-- Logs begin at Mon 2022-07-04 08:35:10 UTC, end at Mon 2022-07-04 09:39:45 UTC. -- Jul 04 09:05:02.621102 dv-cinder-844563-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 04 09:05:02.632100 dv-cinder-844563-4-iscsi devstack@keystone.service[80160]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 04 09:05:02.634149 dv-cinder-844563-4-iscsi devstack@keystone.service[80160]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Jul 4 09:05:02 2022] *** Jul 04 09:05:02.634149 dv-cinder-844563-4-iscsi devstack@keystone.service[80160]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 04 09:05:02.634149 dv-cinder-844563-4-iscsi devstack@keystone.service[80160]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 04 09:05:02.634149 dv-cinder-844563-4-iscsi devstack@keystone.service[80160]: nodename: dv-cinder-844563-4-iscsi Jul 04 09:05:02.634149 dv-cinder-844563-4-iscsi devstack@keystone.service[80160]: machine: x86_64 Jul 04 09:05:02.634149 dv-cinder-844563-4-iscsi devstack@keystone.service[80160]: clock source: unix Jul 04 09:05:02.634149 dv-cinder-844563-4-iscsi devstack@keystone.service[80160]: pcre jit disabled Jul 04 09:05:02.634149 dv-cinder-844563-4-iscsi devstack@keystone.service[80160]: detected number of CPU cores: 4 Jul 04 09:05:02.634149 dv-cinder-844563-4-iscsi devstack@keystone.service[80160]: current working directory: / Jul 04 09:05:02.634149 dv-cinder-844563-4-iscsi devstack@keystone.service[80160]: detected binary path: /usr/bin/uwsgi-core Jul 04 09:05:02.634149 dv-cinder-844563-4-iscsi devstack@keystone.service[80160]: your processes number limit is 47680 Jul 04 09:05:02.634149 dv-cinder-844563-4-iscsi devstack@keystone.service[80160]: your memory page size is 4096 bytes Jul 04 09:05:02.634939 dv-cinder-844563-4-iscsi devstack@keystone.service[80160]: detected max file descriptor number: 2048 Jul 04 09:05:02.634939 dv-cinder-844563-4-iscsi devstack@keystone.service[80160]: lock engine: pthread robust mutexes Jul 04 09:05:02.634939 dv-cinder-844563-4-iscsi devstack@keystone.service[80160]: thunder lock: enabled Jul 04 09:05:02.634939 dv-cinder-844563-4-iscsi devstack@keystone.service[80160]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 04 09:05:02.634939 dv-cinder-844563-4-iscsi devstack@keystone.service[80160]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jul 04 09:05:02.678124 dv-cinder-844563-4-iscsi devstack@keystone.service[80160]: Python main interpreter initialized at 0x558ee5b589c0 Jul 04 09:05:02.678124 dv-cinder-844563-4-iscsi devstack@keystone.service[80160]: python threads support enabled Jul 04 09:05:02.678124 dv-cinder-844563-4-iscsi devstack@keystone.service[80160]: your server socket listen backlog is limited to 100 connections Jul 04 09:05:02.678124 dv-cinder-844563-4-iscsi devstack@keystone.service[80160]: your mercy for graceful operations on workers is 90 seconds Jul 04 09:05:02.678429 dv-cinder-844563-4-iscsi devstack@keystone.service[80160]: mapped 403077 bytes (393 KB) for 2 cores Jul 04 09:05:02.678510 dv-cinder-844563-4-iscsi devstack@keystone.service[80160]: *** Operational MODE: preforking *** Jul 04 09:05:02.678741 dv-cinder-844563-4-iscsi devstack@keystone.service[80160]: *** uWSGI is running in multiple interpreter mode *** Jul 04 09:05:02.678741 dv-cinder-844563-4-iscsi devstack@keystone.service[80160]: spawned uWSGI master process (pid: 80160) Jul 04 09:05:02.678925 dv-cinder-844563-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 04 09:05:02.684790 dv-cinder-844563-4-iscsi devstack@keystone.service[80160]: spawned uWSGI worker 1 (pid: 80170, cores: 1) Jul 04 09:05:02.684790 dv-cinder-844563-4-iscsi devstack@keystone.service[80160]: spawned uWSGI worker 2 (pid: 80171, cores: 1) Jul 04 09:05:02.684790 dv-cinder-844563-4-iscsi devstack@keystone.service[80160]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 04 09:05:08.179441 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 04 09:05:08.179441 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: from cryptography.utils import int_from_bytes Jul 04 09:05:08.180239 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 04 09:05:08.180239 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: from cryptography.utils import int_from_bytes Jul 04 09:05:08.193565 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 04 09:05:08.193565 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: from cryptography.utils import int_from_bytes Jul 04 09:05:08.194052 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 04 09:05:08.194052 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: from cryptography.utils import int_from_bytes Jul 04 09:05:08.230543 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 04 09:05:08.230749 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 04 09:05:08.230949 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] command line args: [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 04 09:05:08.231225 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 04 09:05:08.231407 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ================================================================================ {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 04 09:05:08.231670 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] admin_token = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.231922 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] config_dir = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.232114 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.232301 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] config_source = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.232486 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] debug = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.232688 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.233069 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.233257 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.233513 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.233766 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.234005 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] list_limit = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.234191 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] log_config_append = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.234382 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.234600 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] log_dir = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.234745 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] log_file = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.234983 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.235235 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.235487 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.235736 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.235917 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.236095 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.236344 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.236593 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.236829 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.237086 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.237325 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.237560 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.237808 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.238057 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.238342 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.238835 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.71/identity {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.239020 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] publish_errors = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.239254 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.239501 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.239667 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.239915 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.240151 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.240388 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] standard_threads = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.240623 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.240869 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.241065 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.241249 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] use_journal = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.241428 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] use_json = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.241507 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 04 09:05:08.241608 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] use_stderr = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.241693 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 04 09:05:08.241776 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] use_syslog = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.241850 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] command line args: [] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 04 09:05:08.242007 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.242114 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 04 09:05:08.242263 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.242349 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ================================================================================ {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 04 09:05:08.242506 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.242590 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] admin_token = **** {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.242764 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] config_dir = [] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.242846 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.242954 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.243041 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.243132 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] config_source = [] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.243242 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.243329 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] debug = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.243482 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.243575 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.243723 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.243800 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.243900 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.243984 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.244147 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.244234 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.244394 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.244472 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.244622 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] list_limit = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.244707 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.244827 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] log_config_append = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.244915 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.244988 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.245155 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] log_dir = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.245237 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.245317 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] log_file = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.245391 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.245552 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.245629 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.245796 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.245881 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.246545 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.246629 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.246859 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.247222 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.247303 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.247410 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.247539 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.247619 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.247784 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.247863 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.248043 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.248133 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.248286 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.248411 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.248520 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.248679 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.248758 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.248937 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.249017 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.249201 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.249293 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.249440 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.249524 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.249677 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.249757 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.249920 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.250175 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.250258 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.71/identity {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.250374 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] publish_errors = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.250458 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.250602 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.250687 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.250827 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.250915 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.250989 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.251110 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.251234 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.251362 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.251449 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.251602 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.251687 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] standard_threads = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.251854 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.251932 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.252088 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.252166 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.252314 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.252387 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.252464 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] use_journal = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.252556 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.252638 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] use_json = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.252768 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] use_stderr = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.252855 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.252929 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] use_syslog = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.253005 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.253182 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:05:08.253273 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.253435 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.253524 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.253673 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.253761 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.253937 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.254016 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.254170 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.254256 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.254412 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.254491 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.254644 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.254720 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.254882 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.254967 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.255114 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.255192 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.255359 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.255443 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.255598 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.255676 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.255841 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.255917 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.256076 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.256154 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.256315 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.256393 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.256546 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.256622 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.256791 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.256907 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.257032 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.257154 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.257287 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.257381 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.257532 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.257615 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.257776 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.257842 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.257998 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.258087 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.258230 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.258307 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.258485 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.258570 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.258726 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.258812 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.258965 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.259058 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.259210 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.259294 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.259438 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.259527 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.259674 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.259758 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.259906 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.259990 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.260236 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.260323 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.260405 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.260465 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.260625 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.260712 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.260854 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.260932 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.261106 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.261227 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.261355 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.261435 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.261595 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.262287 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.262581 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.262965 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.263060 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.263229 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.263321 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.263468 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.263562 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.263710 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.263806 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.263942 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.264025 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.264181 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.264273 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.264416 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.264508 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.264648 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.264737 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.264878 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.264961 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.265157 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.265243 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.265403 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.265490 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.265648 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.265734 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.265887 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.265985 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.266122 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.266207 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.266357 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.266449 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.266593 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.266676 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.266832 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.266916 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.267067 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.267151 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.267307 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.267393 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.267535 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.267616 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.267774 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.267859 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.268012 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.268099 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.268254 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.268341 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.268478 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.268561 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.268730 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.268817 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.268959 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.269076 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] auth.external = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.269216 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.269297 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.269439 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.269533 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.269700 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.269787 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.269938 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.270024 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] auth.password = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.270162 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.270246 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] auth.token = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.270389 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.270454 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.270622 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.270708 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.270874 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.270959 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.271102 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.271199 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.271341 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.271423 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.271588 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.271674 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.271820 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.271913 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.272047 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.272129 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] credential.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.272283 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.272369 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.272531 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.272634 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.272769 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.272852 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.272986 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.273090 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.273265 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.273359 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.273493 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.273575 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.273730 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.273823 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.273969 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.274065 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.274208 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.274292 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.274435 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.274521 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.274693 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.274813 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.274961 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.275055 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.275187 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.275275 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.275405 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.275487 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.275630 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.275726 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.275868 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.275953 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.276097 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.276188 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.276328 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.276424 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.276554 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.276690 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.276772 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.276898 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.276980 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.277159 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.277293 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.277463 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.277615 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.277698 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.277887 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.278048 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.278134 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.278281 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.278369 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.278525 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.278613 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.278783 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.278870 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.279022 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.279109 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.279273 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.279368 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.279525 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.279612 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] auth.external = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.279744 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.279826 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.279991 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.280077 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.280242 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.280330 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.280473 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] auth.password = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.280619 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.280753 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] auth.token = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.280895 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.281067 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.281210 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.281344 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.281478 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.281610 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.281752 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.281894 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.282027 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.282160 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.282293 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.282426 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.282564 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.282716 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.282862 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.283001 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] credential.caching = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.283134 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.283272 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.283414 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.283558 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.283693 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.283825 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.283956 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.284087 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.284227 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.284360 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.284492 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.284635 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.284803 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.284987 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.285245 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.285382 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.285514 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.285647 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.285779 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.285910 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.286050 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.286184 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.286322 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.286466 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.286615 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.286759 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.286902 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.287036 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.287168 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.287299 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.287431 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.287562 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] federation.caching = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.287706 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.287839 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.287971 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.288103 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.288234 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.288366 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.288520 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.288660 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.288793 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.288932 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.289084 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.289222 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.289366 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.289499 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.289631 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.289773 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.289905 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.290036 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.290177 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.290321 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.290453 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.290584 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.290715 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.290847 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] identity.caching = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.290987 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.291120 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.291252 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.291383 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.291515 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.291646 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.291787 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.291921 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.292053 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.292200 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.292333 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.292464 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.292608 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.292742 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.292927 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.293137 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.293278 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.293410 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.293557 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.293692 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.293823 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.293956 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.294088 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.294230 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.294371 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.294505 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.294637 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.294775 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.294908 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.295041 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.295182 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.295315 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.295447 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.295595 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.295736 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.295736 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.295736 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.296141 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.296279 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.296424 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.296566 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.296713 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.296853 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.296998 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.297162 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.297302 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.297441 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.297591 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.297731 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.297870 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.298019 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.298159 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.298297 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.298444 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.298585 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.298723 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.298862 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.299007 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.299147 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.299296 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.299437 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.299576 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.299714 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.299863 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.299863 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.300141 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.300283 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.300422 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.300560 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.300699 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.300836 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.300983 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.301169 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.301311 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.301450 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.301589 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.301727 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.301888 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.302035 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.302175 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.302314 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.302453 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.302592 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.302739 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.302879 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.303018 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.303156 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.303295 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.303434 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.303581 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.303721 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.303869 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.304007 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.304146 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.304284 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.304431 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.304571 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.304710 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.304849 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.304988 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.305154 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.305307 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.305447 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.305587 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.305725 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.305874 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.306013 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.306160 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.306300 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.306444 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] role.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.306587 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.306726 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] role.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.306864 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.307012 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.307152 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.307291 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.307429 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.307568 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.307722 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.307722 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.307996 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.308135 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.308273 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.308413 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.309199 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.309313 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.309395 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.309470 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.309545 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.309606 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.309665 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.309747 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.309822 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.309894 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.309966 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.310039 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.310110 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.310192 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.310266 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.310339 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.310411 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.310483 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.310483 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.310624 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.310697 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.310768 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.310840 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.310911 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.311062 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.311139 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.311211 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.311283 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.311355 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.311427 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.311508 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.311582 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.311654 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.311726 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.311797 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.311869 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.311950 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.312024 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.312096 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.312168 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.312240 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.312311 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.312392 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.312466 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.312466 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.312596 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.312668 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.312740 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.312820 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.312894 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.312966 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.313067 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] resource.caching = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.313145 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.313219 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.313304 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.313380 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.313452 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.313524 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.313596 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.313668 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.313752 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.313827 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.313899 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.313971 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.314043 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.314116 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.314196 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.314271 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.314343 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.314415 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.314486 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.314559 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] role.caching = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.314639 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.314713 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] role.driver = sql {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.314785 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.314857 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.314929 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.315001 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.315081 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.315155 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.315228 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.315300 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.315372 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.315444 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.315525 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.315600 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.315672 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.315744 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.315824 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.315898 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.315978 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.316053 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.316125 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.connection = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.316205 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.316287 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.316362 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.316434 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.316515 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.316590 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.316662 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.316735 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.316807 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.316880 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.316880 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.317021 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.317164 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.317242 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.317315 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.317387 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.317474 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.317550 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.317622 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.317695 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.317767 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.317838 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.317920 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.317994 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.318067 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.318139 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.318285 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.318369 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.318533 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.318617 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.318765 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.318845 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.318999 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.319081 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.319231 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.319315 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.319388 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 04 09:05:08.319485 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.319740 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.319986 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.320223 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.320458 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.320687 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.320922 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] token.caching = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.321184 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.321430 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.321666 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.321919 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.322165 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.322409 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.322652 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.322895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.323140 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.323374 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.323610 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.323842 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.324087 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.325348 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.325598 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.325852 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.326153 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.326419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.connection = **** {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.326669 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.326920 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.327161 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.327404 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.327641 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.327877 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.328113 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.328349 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.328595 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.328829 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.329091 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.329330 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.329579 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.329814 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.330057 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.330287 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.330525 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.330759 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:05:08.330930 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 04 09:05:08.351016 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.351253 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.351487 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.357913 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.358144 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.358377 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.360053 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.366724 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.389805 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.390042 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.395573 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.395808 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.535173 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.538470 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.585597 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.585837 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.586713 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.586950 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.652329 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.652759 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.653006 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.653158 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.653267 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.653353 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.653497 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.653581 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.653723 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.653818 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.653951 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.654170 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.678654 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.678887 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.679002 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.679083 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.679158 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.679275 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.679359 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.679501 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.679587 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.679719 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.679801 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.679933 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.680014 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.680145 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.691802 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.692365 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.704658 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.704906 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.705148 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.705295 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.705382 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.705530 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.705614 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.705759 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.705845 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.705981 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.706064 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.706195 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.706405 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.706615 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:05:08.723766 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.723975 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 04 09:05:08.724281 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.724365 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.724494 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 04 09:05:08.724581 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 04 09:05:08.724766 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.724848 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.725013 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 04 09:05:08.725153 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.725308 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.725528 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.725654 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.725792 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.726051 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.726308 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.737165 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.737446 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.737719 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.738006 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.738159 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.738436 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.738705 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.738984 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.746932 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:05:08.748131 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80171) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:05:08.750271 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:05:08.750556 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.750845 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.751133 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.751421 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.751516 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80171) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:05:08.751687 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.751798 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.751961 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.752082 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.752237 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.752359 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.752509 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.752637 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.752785 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.752895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.753072 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.753186 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.753457 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.753724 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.753993 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.754257 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.780305 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.781951 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:05:08.782117 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.782236 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.783726 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80171) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:05:08.784005 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.787612 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:05:08.787850 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.788249 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.789270 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80171) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:05:08.789548 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.789827 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.798450 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80171) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:05:08.798734 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.799748 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:05:08.800042 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.803325 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80171) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:05:08.803612 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.803889 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.804165 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.804437 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.804705 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.804972 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.805262 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.805523 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:05:08.805814 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.806098 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.806384 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.806664 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.806942 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.807213 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.807482 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.824876 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.825179 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.826561 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.826845 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.829807 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80171) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:05:08.831499 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:05:08.833456 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.833751 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.835189 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.835489 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.839172 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80171) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:05:08.841545 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:05:08.843931 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80171) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:05:08.845217 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:05:08.847567 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.847864 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.848150 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.848448 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.848726 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.848896 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.849007 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.849226 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.849525 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.849823 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.850104 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.850397 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.868928 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80171) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:05:08.869234 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.869502 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.869770 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.871654 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:05:08.871953 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.872227 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.872502 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.878201 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.879922 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.880191 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.880475 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.882258 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.882539 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.883304 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80171) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:05:08.883583 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.883978 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.884261 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.884535 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.885858 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:05:08.886142 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.886427 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.886910 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.887365 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.898486 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80171) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:05:08.898764 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.899043 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.899319 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.899595 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.899873 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.900146 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.900790 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:05:08.901100 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.901391 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.901686 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.901969 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.904192 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.904491 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.921275 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80171) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:05:08.923203 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:05:08.924536 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80171) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:05:08.926526 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:05:08.927861 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.929475 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.929909 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.931253 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80171) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:05:08.931532 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.931665 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.931809 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.933348 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:05:08.933780 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.934124 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.939577 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.941239 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80171) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:05:08.941597 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.943245 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:05:08.944453 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.944740 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.945021 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.945320 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.946538 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.946833 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.947227 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.947508 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.955074 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80171) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:05:08.955360 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.955639 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.957026 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:05:08.957344 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.957629 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.963788 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80171) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:05:08.964065 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.964341 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.964609 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.965882 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.966171 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.966456 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.966539 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:05:08.966737 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.966822 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.967017 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.967101 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.967340 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.967431 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.967616 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.967698 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.967909 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.968057 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.968175 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.968324 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.968452 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.968616 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.968901 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.969207 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.969492 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.969761 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.970040 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:08.970319 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:05:09.008239 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x558ee5b589c0 pid: 80171 (default app) Jul 04 09:05:09.009679 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x558ee5b589c0 pid: 80170 (default app) Jul 04 09:05:09.012305 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0640c31-30e4-4985-8514-145efd25e357 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:09.012953 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0640c31-30e4-4985-8514-145efd25e357 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:09.013263 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0640c31-30e4-4985-8514-145efd25e357 None None] PATH_INFO: `/v3` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:09.017474 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1/1] 10.222.0.71 () {50 vars in 791 bytes} [Mon Jul 4 09:05:09 2022] GET /identity/v3/ => generated 255 bytes in 9 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:16.709163 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72ead3e-1bf6-4886-886c-3971c53d68c5 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:16.709706 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72ead3e-1bf6-4886-886c-3971c53d68c5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:16.709803 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72ead3e-1bf6-4886-886c-3971c53d68c5 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:16.714320 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1/2] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:05:16 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:05:16.720284 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae163b-d9c6-4959-ace5-8c491d803724 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:16.720683 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae163b-d9c6-4959-ace5-8c491d803724 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:16.720841 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae163b-d9c6-4959-ace5-8c491d803724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:16.791093 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: INFO dbcounter [None req-62ae163b-d9c6-4959-ace5-8c491d803724 None None] Registered counter for database keystone Jul 04 09:05:16.812229 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG oslo_db.sqlalchemy.engines [None req-62ae163b-d9c6-4959-ace5-8c491d803724 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80171) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 04 09:05:16.818489 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writer thread running {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 04 09:05:17.015809 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG passlib.handlers.bcrypt [None req-62ae163b-d9c6-4959-ace5-8c491d803724 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80171) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 04 09:05:17.016146 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG passlib.handlers.bcrypt [None req-62ae163b-d9c6-4959-ace5-8c491d803724 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80171) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 04 09:05:17.071819 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-62ae163b-d9c6-4959-ace5-8c491d803724 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:05:17.073646 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-62ae163b-d9c6-4959-ace5-8c491d803724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:05:17.199887 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 2/3] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:05:16 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 481 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 04 09:05:17.403951 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72ead3e-1bf6-4886-886c-3971c53d68c5 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:17.474702 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: INFO dbcounter [None req-c72ead3e-1bf6-4886-886c-3971c53d68c5 None None] Registered counter for database keystone Jul 04 09:05:17.494280 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG oslo_db.sqlalchemy.engines [None req-c72ead3e-1bf6-4886-886c-3971c53d68c5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80170) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 04 09:05:17.501250 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writer thread running {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 04 09:05:17.618975 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72ead3e-1bf6-4886-886c-3971c53d68c5 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:17.624029 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7860ae8-ce0b-4508-9fbf-afa4c48d340e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:17.624685 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7860ae8-ce0b-4508-9fbf-afa4c48d340e admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:17.624969 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7860ae8-ce0b-4508-9fbf-afa4c48d340e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:17.625272 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7860ae8-ce0b-4508-9fbf-afa4c48d340e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:17.633328 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7860ae8-ce0b-4508-9fbf-afa4c48d340e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:17.685541 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e7860ae8-ce0b-4508-9fbf-afa4c48d340e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 04 09:05:17.685830 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7860ae8-ce0b-4508-9fbf-afa4c48d340e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:17.695286 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 2/4] 10.222.0.71 () {58 vars in 1324 bytes} [Mon Jul 4 09:05:17 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 09:05:17.701694 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ae163b-d9c6-4959-ace5-8c491d803724 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:17.716740 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ae163b-d9c6-4959-ace5-8c491d803724 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:17.721224 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52423d0d-9ca3-41e6-8086-ceb96034aac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:17.722337 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52423d0d-9ca3-41e6-8086-ceb96034aac2 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:17.722631 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52423d0d-9ca3-41e6-8086-ceb96034aac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:17.722916 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52423d0d-9ca3-41e6-8086-ceb96034aac2 admin admin] PATH_INFO: `/v3/projects/a34f9b2c1cbf4bceb4067648553a5416` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:17.727050 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52423d0d-9ca3-41e6-8086-ceb96034aac2 admin admin] RBAC: Authorizing `identity:get_project(project_id=a34f9b2c1cbf4bceb4067648553a5416)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:17.779009 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: WARNING keystone.common.rbac_enforcer.enforcer [None req-52423d0d-9ca3-41e6-8086-ceb96034aac2 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 04 09:05:17.779357 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52423d0d-9ca3-41e6-8086-ceb96034aac2 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:17.783949 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 3/5] 10.222.0.71 () {56 vars in 1207 bytes} [Mon Jul 4 09:05:17 2022] GET /identity/v3/projects/a34f9b2c1cbf4bceb4067648553a5416 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:19.778031 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a1fa2-21ee-4db5-8fca-771e3de4a248 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:19.778559 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a1fa2-21ee-4db5-8fca-771e3de4a248 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:19.778665 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a1fa2-21ee-4db5-8fca-771e3de4a248 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:19.782722 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 3/6] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:05:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:05:19.788112 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed654748-033b-4913-a2b0-ee217dc1a228 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:19.788516 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed654748-033b-4913-a2b0-ee217dc1a228 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:19.788693 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed654748-033b-4913-a2b0-ee217dc1a228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:19.807300 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-ed654748-033b-4913-a2b0-ee217dc1a228 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:05:19.808903 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ed654748-033b-4913-a2b0-ee217dc1a228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:05:19.816863 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 4/7] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:05:19 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 04 09:05:19.840052 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340a1fa2-21ee-4db5-8fca-771e3de4a248 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:19.851910 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340a1fa2-21ee-4db5-8fca-771e3de4a248 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:19.856323 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccb6067-96a0-44aa-90b3-4f40c49dde42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:19.856966 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb6067-96a0-44aa-90b3-4f40c49dde42 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:19.857285 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb6067-96a0-44aa-90b3-4f40c49dde42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:19.857573 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb6067-96a0-44aa-90b3-4f40c49dde42 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:19.865517 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccb6067-96a0-44aa-90b3-4f40c49dde42 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:19.867320 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccb6067-96a0-44aa-90b3-4f40c49dde42 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:19.877204 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 4/8] 10.222.0.71 () {58 vars in 1324 bytes} [Mon Jul 4 09:05:19 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 09:05:19.882522 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed654748-033b-4913-a2b0-ee217dc1a228 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:19.892337 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed654748-033b-4913-a2b0-ee217dc1a228 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:19.896737 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa923b0-c195-45d2-98ea-eb4aec0d1485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:19.897632 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa923b0-c195-45d2-98ea-eb4aec0d1485 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:19.897924 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa923b0-c195-45d2-98ea-eb4aec0d1485 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:19.898207 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa923b0-c195-45d2-98ea-eb4aec0d1485 admin admin] PATH_INFO: `/v3/users/fa43c0e65110476d9e52a93b45c1a988` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:19.899601 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa923b0-c195-45d2-98ea-eb4aec0d1485 admin admin] RBAC: Authorizing `identity:get_user(user_id=fa43c0e65110476d9e52a93b45c1a988)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:19.901478 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa923b0-c195-45d2-98ea-eb4aec0d1485 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:19.903829 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 5/9] 10.222.0.71 () {56 vars in 1198 bytes} [Mon Jul 4 09:05:19 2022] GET /identity/v3/users/fa43c0e65110476d9e52a93b45c1a988 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:22.012967 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19336106-f23a-4eba-9ffe-948266f2f5ee None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:22.013525 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19336106-f23a-4eba-9ffe-948266f2f5ee None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:22.013630 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19336106-f23a-4eba-9ffe-948266f2f5ee None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:22.014957 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 5/10] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:05:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:05:22.020069 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa344a-fedc-42e4-95fe-827e1788ce8b None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:22.020480 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa344a-fedc-42e4-95fe-827e1788ce8b None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:22.020640 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa344a-fedc-42e4-95fe-827e1788ce8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:22.024575 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368382a6-555a-4bf6-9651-c41fea6d1acf None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:22.024867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368382a6-555a-4bf6-9651-c41fea6d1acf None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:22.025185 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368382a6-555a-4bf6-9651-c41fea6d1acf None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:22.026444 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 6/11] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:05:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:05:22.032107 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe731ea-d0b9-4e6d-a8c2-ed9d7120956e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:22.032400 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe731ea-d0b9-4e6d-a8c2-ed9d7120956e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:22.032687 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe731ea-d0b9-4e6d-a8c2-ed9d7120956e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:22.037878 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-b9fa344a-fedc-42e4-95fe-827e1788ce8b None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:05:22.039070 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b9fa344a-fedc-42e4-95fe-827e1788ce8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:05:22.072313 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG passlib.handlers.bcrypt [None req-efe731ea-d0b9-4e6d-a8c2-ed9d7120956e None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80170) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 04 09:05:22.072730 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG passlib.handlers.bcrypt [None req-efe731ea-d0b9-4e6d-a8c2-ed9d7120956e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80170) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 04 09:05:22.118987 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-efe731ea-d0b9-4e6d-a8c2-ed9d7120956e None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:05:22.120503 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 6/12] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:05:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:05:22.120700 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-efe731ea-d0b9-4e6d-a8c2-ed9d7120956e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:05:22.142901 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa344a-fedc-42e4-95fe-827e1788ce8b None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:22.166509 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:22.221787 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 7/13] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:05:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:05:22.244774 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe731ea-d0b9-4e6d-a8c2-ed9d7120956e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:22.246755 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa344a-fedc-42e4-95fe-827e1788ce8b None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:22.280734 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1eebf6-c714-4a9e-bf7d-e7242574eb7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:22.281369 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1eebf6-c714-4a9e-bf7d-e7242574eb7c None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:22.281661 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1eebf6-c714-4a9e-bf7d-e7242574eb7c None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:22.281996 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1eebf6-c714-4a9e-bf7d-e7242574eb7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:22.346109 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe731ea-d0b9-4e6d-a8c2-ed9d7120956e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:22.348347 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:22.349684 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1eebf6-c714-4a9e-bf7d-e7242574eb7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:22.351195 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1eebf6-c714-4a9e-bf7d-e7242574eb7c None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:22.374470 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:22.386890 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce8ec47-8e5f-4a47-85be-b5c91f3bb4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:22.388024 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce8ec47-8e5f-4a47-85be-b5c91f3bb4bf None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:22.388316 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce8ec47-8e5f-4a47-85be-b5c91f3bb4bf None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:22.388606 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce8ec47-8e5f-4a47-85be-b5c91f3bb4bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:22.401188 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce8ec47-8e5f-4a47-85be-b5c91f3bb4bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:05:22.412801 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-fce8ec47-8e5f-4a47-85be-b5c91f3bb4bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:05:22.412801 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:22.412801 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:22.412801 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:22.412801 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:22.412801 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:22.412801 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:22.412801 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:22.412801 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:22.412801 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:22.412801 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:22.412801 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:22.412801 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:05:22.412801 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:05:22.412801 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:05:22.412801 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:05:22.413850 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:05:22.413850 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:05:22.413850 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:05:22.413850 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:05:22.413850 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:22.413850 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:22.413850 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:22.413850 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:22.413850 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:22.413850 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:22.413850 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:22.413850 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:22.413850 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:22.413850 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:22.413850 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:22.413850 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:22.414818 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:22.414818 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:22.414818 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:22.414818 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:22.414818 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:05:22.414818 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:05:22.414818 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:05:22.414818 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:05:22.414818 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:05:22.414818 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:05:22.414818 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:05:22.414818 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:05:22.414818 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 8/14] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 4 09:05:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:05:22.419783 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce8ec47-8e5f-4a47-85be-b5c91f3bb4bf None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:22.454019 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 7/15] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:05:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 04 09:05:22.459754 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1eebf6-c714-4a9e-bf7d-e7242574eb7c None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:22.523186 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce8ec47-8e5f-4a47-85be-b5c91f3bb4bf None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:22.556128 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25615b98-88e0-4a1d-9f0d-cc7da98fc9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:22.556589 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1eebf6-c714-4a9e-bf7d-e7242574eb7c None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:22.556710 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25615b98-88e0-4a1d-9f0d-cc7da98fc9b9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:22.556996 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25615b98-88e0-4a1d-9f0d-cc7da98fc9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:22.557309 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25615b98-88e0-4a1d-9f0d-cc7da98fc9b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:22.558706 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25615b98-88e0-4a1d-9f0d-cc7da98fc9b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:22.559017 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25615b98-88e0-4a1d-9f0d-cc7da98fc9b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:22.560497 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25615b98-88e0-4a1d-9f0d-cc7da98fc9b9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:22.572472 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 9/16] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:05:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:05:22.578542 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25615b98-88e0-4a1d-9f0d-cc7da98fc9b9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:22.579808 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:22.592275 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67248b70-2d5d-402f-94e3-99cf48616ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:22.593102 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67248b70-2d5d-402f-94e3-99cf48616ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:22.593394 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67248b70-2d5d-402f-94e3-99cf48616ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:22.593677 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67248b70-2d5d-402f-94e3-99cf48616ac7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:22.603640 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67248b70-2d5d-402f-94e3-99cf48616ac7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:22.605367 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67248b70-2d5d-402f-94e3-99cf48616ac7 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:22.606228 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:22.628174 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-67248b70-2d5d-402f-94e3-99cf48616ac7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:05:22.628174 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:22.628174 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:22.628174 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:22.628174 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:22.628174 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:22.628174 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:22.628174 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:22.628174 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:22.628174 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:22.628174 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:22.628174 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:22.628174 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:05:22.628174 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:05:22.628174 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:05:22.629075 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:05:22.629075 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:22.629075 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:22.629075 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:22.629075 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:22.629075 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:22.629075 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:22.629075 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:22.629075 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:22.629075 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:22.629075 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:22.629075 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:22.629075 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:22.629075 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:22.629075 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:22.629075 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:22.629075 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:22.629900 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:05:22.629900 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:05:22.629900 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:05:22.629900 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:05:22.629900 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 8/17] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:05:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:05:22.635419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67248b70-2d5d-402f-94e3-99cf48616ac7 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:22.686751 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25615b98-88e0-4a1d-9f0d-cc7da98fc9b9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:22.719885 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d51f24-83fe-4aec-96d5-2839fc2a7993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:22.720758 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d51f24-83fe-4aec-96d5-2839fc2a7993 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:22.721071 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d51f24-83fe-4aec-96d5-2839fc2a7993 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:22.721360 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d51f24-83fe-4aec-96d5-2839fc2a7993 None admin] PATH_INFO: `/v3/users/fa43c0e65110476d9e52a93b45c1a988` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:22.722921 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d51f24-83fe-4aec-96d5-2839fc2a7993 None admin] RBAC: Authorizing `identity:get_user(user_id=fa43c0e65110476d9e52a93b45c1a988)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:22.724372 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d51f24-83fe-4aec-96d5-2839fc2a7993 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:22.726015 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 10/18] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 4 09:05:22 2022] GET /identity/v3/users/fa43c0e65110476d9e52a93b45c1a988 => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:05:22.732197 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d51f24-83fe-4aec-96d5-2839fc2a7993 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:22.736144 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67248b70-2d5d-402f-94e3-99cf48616ac7 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:22.771094 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae25fa-a1c2-48f1-890b-6c69a0d9c15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:22.771762 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae25fa-a1c2-48f1-890b-6c69a0d9c15e None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:22.772049 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae25fa-a1c2-48f1-890b-6c69a0d9c15e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:22.772332 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae25fa-a1c2-48f1-890b-6c69a0d9c15e None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:22.773088 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ae25fa-a1c2-48f1-890b-6c69a0d9c15e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:22.773409 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ae25fa-a1c2-48f1-890b-6c69a0d9c15e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:22.774800 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ae25fa-a1c2-48f1-890b-6c69a0d9c15e None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:22.791491 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 9/19] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:05:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:05:22.830671 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:22.837976 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d51f24-83fe-4aec-96d5-2839fc2a7993 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:22.875184 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c53d3a-c1c9-426d-baa9-2b2f49847704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:22.875992 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c53d3a-c1c9-426d-baa9-2b2f49847704 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:22.876280 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c53d3a-c1c9-426d-baa9-2b2f49847704 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:22.876558 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c53d3a-c1c9-426d-baa9-2b2f49847704 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:22.877812 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c53d3a-c1c9-426d-baa9-2b2f49847704 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:22.879280 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c53d3a-c1c9-426d-baa9-2b2f49847704 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:22.880909 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 11/20] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:05:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:05:22.887308 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae25fa-a1c2-48f1-890b-6c69a0d9c15e None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:22.922977 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:22.991178 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae25fa-a1c2-48f1-890b-6c69a0d9c15e None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:23.029479 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c156cd3-babd-43d0-9da6-848af59834a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:23.030828 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c156cd3-babd-43d0-9da6-848af59834a9 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:23.031283 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c156cd3-babd-43d0-9da6-848af59834a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:23.031741 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c156cd3-babd-43d0-9da6-848af59834a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:23.032607 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c156cd3-babd-43d0-9da6-848af59834a9 None admin] RBAC: Adding query filter params (role.id=f36e053b442540f08041ac7878760f65, scope.domain.id=default, user.id=fa43c0e65110476d9e52a93b45c1a988) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:23.032925 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c156cd3-babd-43d0-9da6-848af59834a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:23.034413 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c156cd3-babd-43d0-9da6-848af59834a9 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:23.041593 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 10/21] 10.222.0.71 () {56 vars in 1322 bytes} [Mon Jul 4 09:05:22 2022] GET /identity/v3/role_assignments?user.id=fa43c0e65110476d9e52a93b45c1a988&scope.domain.id=default&role.id=f36e053b442540f08041ac7878760f65 => generated 233 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:25.297276 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5661bfc3-1c11-40bd-9080-a99d7a6b6e60 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:25.298294 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5661bfc3-1c11-40bd-9080-a99d7a6b6e60 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:25.298786 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5661bfc3-1c11-40bd-9080-a99d7a6b6e60 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:25.300680 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 12/22] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:05:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:05:25.305833 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2413ae0-420b-46b1-a002-7f6d7a167afe None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:25.306683 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2413ae0-420b-46b1-a002-7f6d7a167afe None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:25.307167 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2413ae0-420b-46b1-a002-7f6d7a167afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:25.324443 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b82c7f-4126-4320-83bc-a3638ca42ef3 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:25.325110 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b82c7f-4126-4320-83bc-a3638ca42ef3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:25.325591 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b82c7f-4126-4320-83bc-a3638ca42ef3 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:25.327152 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 13/23] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:05:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:05:25.332228 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f20568-d212-4ecd-89bd-47590e768d70 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:25.332776 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f20568-d212-4ecd-89bd-47590e768d70 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:25.333323 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f20568-d212-4ecd-89bd-47590e768d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:25.336079 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-d2413ae0-420b-46b1-a002-7f6d7a167afe None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:05:25.337802 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d2413ae0-420b-46b1-a002-7f6d7a167afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:05:25.366114 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f0f20568-d212-4ecd-89bd-47590e768d70 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:05:25.368369 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f0f20568-d212-4ecd-89bd-47590e768d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:05:25.507392 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 11/24] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:05:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:05:25.526504 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 14/25] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:05:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:05:25.546233 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2413ae0-420b-46b1-a002-7f6d7a167afe None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:25.568124 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f20568-d212-4ecd-89bd-47590e768d70 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:25.606684 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:25.696345 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2413ae0-420b-46b1-a002-7f6d7a167afe None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:25.709404 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:25.714794 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f20568-d212-4ecd-89bd-47590e768d70 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:25.732712 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e57db9d-d33e-4807-a5e7-66349ea3fb50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:25.733769 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e57db9d-d33e-4807-a5e7-66349ea3fb50 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:25.734271 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e57db9d-d33e-4807-a5e7-66349ea3fb50 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:25.734723 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e57db9d-d33e-4807-a5e7-66349ea3fb50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:25.745163 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e57db9d-d33e-4807-a5e7-66349ea3fb50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:25.746919 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e57db9d-d33e-4807-a5e7-66349ea3fb50 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:25.751351 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872fe673-ce10-4f01-859f-a9cde0d181db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:25.752235 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872fe673-ce10-4f01-859f-a9cde0d181db None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:25.752668 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872fe673-ce10-4f01-859f-a9cde0d181db None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:25.753128 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872fe673-ce10-4f01-859f-a9cde0d181db None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:25.754077 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872fe673-ce10-4f01-859f-a9cde0d181db None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:25.755597 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872fe673-ce10-4f01-859f-a9cde0d181db None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:25.766392 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-3e57db9d-d33e-4807-a5e7-66349ea3fb50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:05:25.766392 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:25.766392 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:25.766392 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:25.766392 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:25.766392 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:25.766392 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:25.766392 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:25.766392 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:25.766392 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:25.766392 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:25.766392 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:25.766392 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:05:25.766392 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:05:25.766392 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:05:25.767314 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:05:25.767314 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:25.767314 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:25.767314 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:25.767314 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:25.767314 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:25.767314 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:25.767314 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:25.767314 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:25.767314 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:25.767314 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:25.767314 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:25.767314 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:25.767314 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:25.767314 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:25.767314 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:25.767314 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:25.768166 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:05:25.768166 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:05:25.768166 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:05:25.768166 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:05:25.768166 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 12/26] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:05:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:05:25.769322 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06540a79-6d97-4bd3-b7e4-195eb75d52a4 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:25.769691 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06540a79-6d97-4bd3-b7e4-195eb75d52a4 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:25.770135 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06540a79-6d97-4bd3-b7e4-195eb75d52a4 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:25.771707 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 13/27] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:05:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:05:25.772880 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d59e9-188b-4322-bf67-2aa0cee36606 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:25.773320 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d59e9-188b-4322-bf67-2aa0cee36606 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:25.773704 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d59e9-188b-4322-bf67-2aa0cee36606 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:25.779747 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 14/28] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:05:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:05:25.779878 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3a9ff-68cb-4ae4-95f4-88b97233cf1b None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:25.780240 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3a9ff-68cb-4ae4-95f4-88b97233cf1b None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:25.780363 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3a9ff-68cb-4ae4-95f4-88b97233cf1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:25.797433 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-ccd3a9ff-68cb-4ae4-95f4-88b97233cf1b None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:05:25.798676 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ccd3a9ff-68cb-4ae4-95f4-88b97233cf1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:05:25.867383 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 15/29] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:05:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:05:25.868166 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd3a9ff-68cb-4ae4-95f4-88b97233cf1b None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:25.885434 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:25.895597 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 15/30] 10.222.0.71 () {60 vars in 1131 bytes} [Mon Jul 4 09:05:25 2022] POST /identity/v3/roles => generated 219 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:05:25.902556 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78eccd2-690b-410b-8eba-f32f35423cc5 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:25.903158 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78eccd2-690b-410b-8eba-f32f35423cc5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:25.903929 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78eccd2-690b-410b-8eba-f32f35423cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:25.942225 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b78eccd2-690b-410b-8eba-f32f35423cc5 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:05:25.944298 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b78eccd2-690b-410b-8eba-f32f35423cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:05:25.979261 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd3a9ff-68cb-4ae4-95f4-88b97233cf1b None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:26.018422 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafce9c2-b45f-4264-a17b-2997c8fa3bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:26.019107 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafce9c2-b45f-4264-a17b-2997c8fa3bf5 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:26.019391 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafce9c2-b45f-4264-a17b-2997c8fa3bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:26.019672 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafce9c2-b45f-4264-a17b-2997c8fa3bf5 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:26.020402 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafce9c2-b45f-4264-a17b-2997c8fa3bf5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:26.020715 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafce9c2-b45f-4264-a17b-2997c8fa3bf5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:26.022639 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafce9c2-b45f-4264-a17b-2997c8fa3bf5 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:26.036697 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 16/31] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:05:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:26.037601 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafce9c2-b45f-4264-a17b-2997c8fa3bf5 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:26.119851 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 16/32] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:05:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:05:26.121138 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78eccd2-690b-410b-8eba-f32f35423cc5 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:26.123363 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:26.157104 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafce9c2-b45f-4264-a17b-2997c8fa3bf5 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:26.204379 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea1945f-b0b3-4f31-806f-111dfd26167a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:26.206217 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea1945f-b0b3-4f31-806f-111dfd26167a None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:26.206733 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea1945f-b0b3-4f31-806f-111dfd26167a None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:26.207207 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea1945f-b0b3-4f31-806f-111dfd26167a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:26.228771 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea1945f-b0b3-4f31-806f-111dfd26167a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:05:26.241406 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-3ea1945f-b0b3-4f31-806f-111dfd26167a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:05:26.241406 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:26.241406 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:26.241406 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:26.241406 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:26.241406 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:26.241406 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:26.241406 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:26.241406 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:26.241406 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:26.241406 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:26.241406 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:26.241406 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:05:26.241406 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:05:26.241406 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:05:26.241406 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:05:26.242390 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:05:26.242390 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:05:26.242390 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:05:26.242390 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:05:26.242390 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:26.242390 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:26.242390 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:26.242390 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:26.242390 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:26.242390 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:26.242390 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:26.242390 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:26.242390 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:26.242390 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:26.242390 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:26.242390 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:26.243189 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:26.243189 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:26.243189 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:26.243189 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:26.243189 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:05:26.243189 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:05:26.243189 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:05:26.243189 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:05:26.243189 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:05:26.243189 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:05:26.243189 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:05:26.243189 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:05:26.246568 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 17/33] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 4 09:05:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:05:26.247609 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea1945f-b0b3-4f31-806f-111dfd26167a None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:26.252715 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:26.281855 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78eccd2-690b-410b-8eba-f32f35423cc5 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:26.328967 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0065539d-a1b7-49e5-bc7c-91205ba0b368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:26.329891 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0065539d-a1b7-49e5-bc7c-91205ba0b368 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:26.330496 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0065539d-a1b7-49e5-bc7c-91205ba0b368 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:26.330852 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0065539d-a1b7-49e5-bc7c-91205ba0b368 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:26.331702 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0065539d-a1b7-49e5-bc7c-91205ba0b368 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:26.334263 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0065539d-a1b7-49e5-bc7c-91205ba0b368 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:26.387084 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 17/34] 10.222.0.71 () {60 vars in 1141 bytes} [Mon Jul 4 09:05:26 2022] POST /identity/v3/projects => generated 301 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:05:26.388368 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0065539d-a1b7-49e5-bc7c-91205ba0b368 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:26.404584 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea1945f-b0b3-4f31-806f-111dfd26167a None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:26.441355 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:26.450243 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9d3311-2220-4f38-af28-4aaf31a07e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:26.451409 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d3311-2220-4f38-af28-4aaf31a07e29 None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:26.451906 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d3311-2220-4f38-af28-4aaf31a07e29 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:26.452392 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d3311-2220-4f38-af28-4aaf31a07e29 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:26.453499 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9d3311-2220-4f38-af28-4aaf31a07e29 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:26.455535 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9d3311-2220-4f38-af28-4aaf31a07e29 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:26.516696 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0065539d-a1b7-49e5-bc7c-91205ba0b368 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:26.554142 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c919305d-f073-4b3b-9e22-c3f7d84cc6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:26.554944 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c919305d-f073-4b3b-9e22-c3f7d84cc6a7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:26.555237 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c919305d-f073-4b3b-9e22-c3f7d84cc6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:26.555525 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c919305d-f073-4b3b-9e22-c3f7d84cc6a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:26.556348 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c919305d-f073-4b3b-9e22-c3f7d84cc6a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:26.556706 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c919305d-f073-4b3b-9e22-c3f7d84cc6a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:26.560640 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c919305d-f073-4b3b-9e22-c3f7d84cc6a7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:26.561323 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:26.573139 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 18/35] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:05:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:26.578481 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c919305d-f073-4b3b-9e22-c3f7d84cc6a7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:26.679272 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 18/36] 10.222.0.71 () {60 vars in 1131 bytes} [Mon Jul 4 09:05:26 2022] POST /identity/v3/roles => generated 225 bytes in 432 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:05:26.680938 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff596f-cfea-41fd-aa57-bfcb4ec9518b None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:26.689701 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff596f-cfea-41fd-aa57-bfcb4ec9518b None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:26.690100 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff596f-cfea-41fd-aa57-bfcb4ec9518b None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:26.691696 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 19/37] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:05:26 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:05:26.700723 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831bf88-04e0-4b06-babc-49674bf946a9 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:26.701082 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831bf88-04e0-4b06-babc-49674bf946a9 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:26.701375 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831bf88-04e0-4b06-babc-49674bf946a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:26.703129 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c919305d-f073-4b3b-9e22-c3f7d84cc6a7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:26.732072 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-9831bf88-04e0-4b06-babc-49674bf946a9 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:05:26.734316 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-9831bf88-04e0-4b06-babc-49674bf946a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:05:26.750925 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3ae237-54e3-4b90-ba89-4a48bcc495f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:26.752751 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ae237-54e3-4b90-ba89-4a48bcc495f1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:26.753307 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ae237-54e3-4b90-ba89-4a48bcc495f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:26.753728 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ae237-54e3-4b90-ba89-4a48bcc495f1 None admin] PATH_INFO: `/v3/users/fa43c0e65110476d9e52a93b45c1a988` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:26.755373 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3ae237-54e3-4b90-ba89-4a48bcc495f1 None admin] RBAC: Authorizing `identity:get_user(user_id=fa43c0e65110476d9e52a93b45c1a988)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:26.757236 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3ae237-54e3-4b90-ba89-4a48bcc495f1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:26.759486 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 19/38] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 4 09:05:26 2022] GET /identity/v3/users/fa43c0e65110476d9e52a93b45c1a988 => generated 247 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:26.766172 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3ae237-54e3-4b90-ba89-4a48bcc495f1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:26.835892 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 20/39] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:05:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:05:26.865704 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9831bf88-04e0-4b06-babc-49674bf946a9 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:26.866214 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:26.923494 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3ae237-54e3-4b90-ba89-4a48bcc495f1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:26.966783 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248fb096-e6ea-4bea-9975-1cb0b7e6f011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:26.967589 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248fb096-e6ea-4bea-9975-1cb0b7e6f011 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:26.967870 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248fb096-e6ea-4bea-9975-1cb0b7e6f011 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:26.968157 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248fb096-e6ea-4bea-9975-1cb0b7e6f011 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:26.969394 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248fb096-e6ea-4bea-9975-1cb0b7e6f011 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:26.970857 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248fb096-e6ea-4bea-9975-1cb0b7e6f011 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:26.972493 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 20/40] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:05:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:26.978039 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248fb096-e6ea-4bea-9975-1cb0b7e6f011 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:27.005279 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9831bf88-04e0-4b06-babc-49674bf946a9 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:27.091033 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248fb096-e6ea-4bea-9975-1cb0b7e6f011 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:27.122458 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:27.128222 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04de93b2-c804-4d29-832f-59c60bae6a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:27.129153 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de93b2-c804-4d29-832f-59c60bae6a71 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:27.129447 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de93b2-c804-4d29-832f-59c60bae6a71 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:27.129740 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de93b2-c804-4d29-832f-59c60bae6a71 None admin] PATH_INFO: `/v3/domains/default/users/fa43c0e65110476d9e52a93b45c1a988/roles/f36e053b442540f08041ac7878760f65` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:27.131787 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04de93b2-c804-4d29-832f-59c60bae6a71 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=fa43c0e65110476d9e52a93b45c1a988, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:27.133372 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04de93b2-c804-4d29-832f-59c60bae6a71 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:27.142232 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:27.149665 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 21/41] 10.222.0.71 () {58 vars in 1361 bytes} [Mon Jul 4 09:05:26 2022] PUT /identity/v3/domains/default/users/fa43c0e65110476d9e52a93b45c1a988/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:05:27.150940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d2862-b815-4634-b6a8-e80b70c53955 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:27.151234 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d2862-b815-4634-b6a8-e80b70c53955 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:27.151620 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d2862-b815-4634-b6a8-e80b70c53955 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:27.153043 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 22/42] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:05:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:05:27.159673 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a438778-2c1b-4322-86c3-249cb2f2bdd6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:27.159997 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a438778-2c1b-4322-86c3-249cb2f2bdd6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:27.160253 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a438778-2c1b-4322-86c3-249cb2f2bdd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:27.170962 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00233534-05c1-4829-8ba1-752456ca0aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:27.171768 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00233534-05c1-4829-8ba1-752456ca0aa0 None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:27.172001 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00233534-05c1-4829-8ba1-752456ca0aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:27.172278 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00233534-05c1-4829-8ba1-752456ca0aa0 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:27.173952 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00233534-05c1-4829-8ba1-752456ca0aa0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:27.174702 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00233534-05c1-4829-8ba1-752456ca0aa0 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:27.187302 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3a438778-2c1b-4322-86c3-249cb2f2bdd6 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:05:27.188534 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3a438778-2c1b-4322-86c3-249cb2f2bdd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:05:27.190275 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 21/43] 10.222.0.71 () {60 vars in 1131 bytes} [Mon Jul 4 09:05:26 2022] POST /identity/v3/roles => generated 223 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:05:27.274431 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 23/44] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:05:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:05:27.298372 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00233534-05c1-4829-8ba1-752456ca0aa0 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:27.421787 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00233534-05c1-4829-8ba1-752456ca0aa0 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:27.456125 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570af81e-3ed0-463e-b56a-1c7fabc6a98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:27.456928 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570af81e-3ed0-463e-b56a-1c7fabc6a98e None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:27.457248 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570af81e-3ed0-463e-b56a-1c7fabc6a98e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:27.457531 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570af81e-3ed0-463e-b56a-1c7fabc6a98e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:27.458707 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570af81e-3ed0-463e-b56a-1c7fabc6a98e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:27.460154 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570af81e-3ed0-463e-b56a-1c7fabc6a98e None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:27.461814 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 22/45] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:05:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:27.467707 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a438778-2c1b-4322-86c3-249cb2f2bdd6 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:27.564916 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a438778-2c1b-4322-86c3-249cb2f2bdd6 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:27.586953 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:27.599569 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6764690c-8bce-4272-af4c-478f1d5d57c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:27.600239 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6764690c-8bce-4272-af4c-478f1d5d57c7 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:27.600528 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6764690c-8bce-4272-af4c-478f1d5d57c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:27.600808 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6764690c-8bce-4272-af4c-478f1d5d57c7 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:27.601564 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6764690c-8bce-4272-af4c-478f1d5d57c7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:27.603005 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6764690c-8bce-4272-af4c-478f1d5d57c7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:27.627612 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 24/46] 10.222.0.71 () {60 vars in 1141 bytes} [Mon Jul 4 09:05:27 2022] POST /identity/v3/projects => generated 312 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:05:29.241645 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b425d113-eadf-4eb9-8087-a679fa243a44 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:29.242169 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b425d113-eadf-4eb9-8087-a679fa243a44 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:29.242251 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b425d113-eadf-4eb9-8087-a679fa243a44 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:29.243594 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 23/47] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:05:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:05:29.249353 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f09c7-14ff-4641-8484-e9dd13419d81 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:29.249758 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f09c7-14ff-4641-8484-e9dd13419d81 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:29.249960 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f09c7-14ff-4641-8484-e9dd13419d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:29.267035 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-613f09c7-14ff-4641-8484-e9dd13419d81 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:05:29.268228 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-613f09c7-14ff-4641-8484-e9dd13419d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:05:29.333434 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 25/48] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:05:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:05:29.357997 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b425d113-eadf-4eb9-8087-a679fa243a44 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:29.412293 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:29.468555 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b425d113-eadf-4eb9-8087-a679fa243a44 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:29.502347 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6024de82-fbe5-493d-9b35-8c800644fdb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:29.503063 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024de82-fbe5-493d-9b35-8c800644fdb0 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:29.503348 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024de82-fbe5-493d-9b35-8c800644fdb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:29.503628 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024de82-fbe5-493d-9b35-8c800644fdb0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:29.514652 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6024de82-fbe5-493d-9b35-8c800644fdb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:05:29.522209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-6024de82-fbe5-493d-9b35-8c800644fdb0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:05:29.522209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:29.522209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:29.522209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:29.522209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:29.522209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:29.522209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:29.522209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:29.522209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:29.522209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:29.522209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:29.522209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:29.522209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:05:29.522209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:05:29.522209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:05:29.522209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:05:29.523099 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:05:29.523099 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:05:29.523099 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:05:29.523099 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:05:29.523099 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:29.523099 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:29.523099 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:29.523099 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:29.523099 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:29.523099 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:29.523099 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:29.523099 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:29.523099 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:29.523099 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:29.523099 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:29.523099 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:29.523934 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:29.523934 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:29.523934 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:29.523934 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:29.523934 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:05:29.523934 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:05:29.523934 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:05:29.523934 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:05:29.523934 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:05:29.523934 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:05:29.523934 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:05:29.523934 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:05:29.523934 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 24/49] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 4 09:05:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:05:29.531672 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613f09c7-14ff-4641-8484-e9dd13419d81 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:29.632446 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613f09c7-14ff-4641-8484-e9dd13419d81 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:29.634777 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:29.664122 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d337141-a481-4b64-9e5d-0333bf7d353a None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:29.664419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d337141-a481-4b64-9e5d-0333bf7d353a None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:29.664709 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d337141-a481-4b64-9e5d-0333bf7d353a None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:29.666128 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 25/50] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:05:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:05:29.668687 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a0b159-9669-458c-b0e7-e5f5f39989e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:29.669384 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a0b159-9669-458c-b0e7-e5f5f39989e9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:29.669671 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a0b159-9669-458c-b0e7-e5f5f39989e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:29.669954 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a0b159-9669-458c-b0e7-e5f5f39989e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:29.670710 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a0b159-9669-458c-b0e7-e5f5f39989e9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:29.671020 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a0b159-9669-458c-b0e7-e5f5f39989e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:29.671196 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25930ef3-3836-4fad-87c8-20ffe0b8411f None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:29.671485 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25930ef3-3836-4fad-87c8-20ffe0b8411f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:29.671767 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25930ef3-3836-4fad-87c8-20ffe0b8411f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:29.672412 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a0b159-9669-458c-b0e7-e5f5f39989e9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:29.681290 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 26/51] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:05:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:29.687108 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a0b159-9669-458c-b0e7-e5f5f39989e9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:29.690328 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-25930ef3-3836-4fad-87c8-20ffe0b8411f None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:05:29.691528 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-25930ef3-3836-4fad-87c8-20ffe0b8411f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:05:29.757509 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 26/52] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:05:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:05:29.782606 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25930ef3-3836-4fad-87c8-20ffe0b8411f None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:29.786055 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a0b159-9669-458c-b0e7-e5f5f39989e9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:29.818658 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522bbbcc-b2ff-4bce-8906-9ae2dd1152f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:29.819535 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522bbbcc-b2ff-4bce-8906-9ae2dd1152f0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:29.819827 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522bbbcc-b2ff-4bce-8906-9ae2dd1152f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:29.820112 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522bbbcc-b2ff-4bce-8906-9ae2dd1152f0 None admin] PATH_INFO: `/v3/users/fa43c0e65110476d9e52a93b45c1a988` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:29.821362 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522bbbcc-b2ff-4bce-8906-9ae2dd1152f0 None admin] RBAC: Authorizing `identity:get_user(user_id=fa43c0e65110476d9e52a93b45c1a988)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:29.822729 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:29.822843 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522bbbcc-b2ff-4bce-8906-9ae2dd1152f0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:29.825439 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 27/53] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 4 09:05:29 2022] GET /identity/v3/users/fa43c0e65110476d9e52a93b45c1a988 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:05:29.830684 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522bbbcc-b2ff-4bce-8906-9ae2dd1152f0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:29.864443 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:29.883535 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25930ef3-3836-4fad-87c8-20ffe0b8411f None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:29.917439 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cecf92-ec44-4672-ba19-4d9b6ed92cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:29.918244 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cecf92-ec44-4672-ba19-4d9b6ed92cd9 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:29.918541 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cecf92-ec44-4672-ba19-4d9b6ed92cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:29.918824 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cecf92-ec44-4672-ba19-4d9b6ed92cd9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:29.920022 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cecf92-ec44-4672-ba19-4d9b6ed92cd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:29.921534 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cecf92-ec44-4672-ba19-4d9b6ed92cd9 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:29.923325 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 27/54] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:05:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:05:29.928673 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cecf92-ec44-4672-ba19-4d9b6ed92cd9 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:29.930679 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522bbbcc-b2ff-4bce-8906-9ae2dd1152f0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:29.964032 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6318d3-5c9f-43c3-a60a-f167f7ab61d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:29.964814 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6318d3-5c9f-43c3-a60a-f167f7ab61d9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:29.965130 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6318d3-5c9f-43c3-a60a-f167f7ab61d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:29.965411 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6318d3-5c9f-43c3-a60a-f167f7ab61d9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:29.966581 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6318d3-5c9f-43c3-a60a-f167f7ab61d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:29.968053 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6318d3-5c9f-43c3-a60a-f167f7ab61d9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:29.969685 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 28/55] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:05:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:05:29.975200 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6318d3-5c9f-43c3-a60a-f167f7ab61d9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:30.030849 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cecf92-ec44-4672-ba19-4d9b6ed92cd9 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:30.038941 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:30.071186 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f644953-b6e5-4911-8f91-694974b30d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:30.071961 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f644953-b6e5-4911-8f91-694974b30d2c None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:30.072265 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f644953-b6e5-4911-8f91-694974b30d2c None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:30.072603 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f644953-b6e5-4911-8f91-694974b30d2c None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:30.074260 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f644953-b6e5-4911-8f91-694974b30d2c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:30.075969 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f644953-b6e5-4911-8f91-694974b30d2c None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:30.105120 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 28/56] 10.222.0.71 () {60 vars in 1140 bytes} [Mon Jul 4 09:05:29 2022] POST /identity/v3/projects => generated 298 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:05:30.188591 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6318d3-5c9f-43c3-a60a-f167f7ab61d9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:30.190242 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:30.227110 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2e5a20-69e3-4649-9ba8-cdec834ad980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:30.227779 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e5a20-69e3-4649-9ba8-cdec834ad980 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:30.228068 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e5a20-69e3-4649-9ba8-cdec834ad980 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:30.228349 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e5a20-69e3-4649-9ba8-cdec834ad980 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:30.229229 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2e5a20-69e3-4649-9ba8-cdec834ad980 None admin] RBAC: Adding query filter params (role.id=f36e053b442540f08041ac7878760f65, scope.domain.id=default, user.id=fa43c0e65110476d9e52a93b45c1a988) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:30.229753 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2e5a20-69e3-4649-9ba8-cdec834ad980 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:30.231846 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2e5a20-69e3-4649-9ba8-cdec834ad980 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:30.240698 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 29/57] 10.222.0.71 () {56 vars in 1322 bytes} [Mon Jul 4 09:05:29 2022] GET /identity/v3/role_assignments?user.id=fa43c0e65110476d9e52a93b45c1a988&scope.domain.id=default&role.id=f36e053b442540f08041ac7878760f65 => generated 530 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:05:32.185827 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968042b0-9435-45ac-b9e7-0ea140854f47 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:32.186350 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968042b0-9435-45ac-b9e7-0ea140854f47 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:32.186436 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968042b0-9435-45ac-b9e7-0ea140854f47 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:32.187749 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 29/58] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:05:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:05:32.192723 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d28083-3277-48ec-8e20-03362ab9676b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:32.193160 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d28083-3277-48ec-8e20-03362ab9676b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:32.193338 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d28083-3277-48ec-8e20-03362ab9676b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:32.210730 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-76d28083-3277-48ec-8e20-03362ab9676b None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:05:32.211953 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-76d28083-3277-48ec-8e20-03362ab9676b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:05:32.277560 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 30/59] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:05:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:05:32.301523 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968042b0-9435-45ac-b9e7-0ea140854f47 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:32.410342 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968042b0-9435-45ac-b9e7-0ea140854f47 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:32.445739 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615574a7-0746-439d-b570-a33d0b3fe2aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:32.446464 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615574a7-0746-439d-b570-a33d0b3fe2aa None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:32.446751 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615574a7-0746-439d-b570-a33d0b3fe2aa None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:32.447032 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615574a7-0746-439d-b570-a33d0b3fe2aa None admin] PATH_INFO: `/v3/domains/default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:32.448262 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615574a7-0746-439d-b570-a33d0b3fe2aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:32.449760 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615574a7-0746-439d-b570-a33d0b3fe2aa None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:32.451391 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 30/60] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:05:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:32.456837 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d28083-3277-48ec-8e20-03362ab9676b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:32.548526 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:32.555181 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d28083-3277-48ec-8e20-03362ab9676b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:32.588940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd8df92-51b3-4afa-a55f-324fac700e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:32.589626 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd8df92-51b3-4afa-a55f-324fac700e51 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:32.589913 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd8df92-51b3-4afa-a55f-324fac700e51 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:32.590195 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd8df92-51b3-4afa-a55f-324fac700e51 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:32.590916 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd8df92-51b3-4afa-a55f-324fac700e51 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:32.592344 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd8df92-51b3-4afa-a55f-324fac700e51 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:32.616470 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 31/61] 10.222.0.71 () {60 vars in 1132 bytes} [Mon Jul 4 09:05:32 2022] POST /identity/v3/users => generated 275 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:05:35.649311 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1526a031-bcfc-46ec-aefa-692b6ae62721 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:35.650338 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1526a031-bcfc-46ec-aefa-692b6ae62721 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:35.650839 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1526a031-bcfc-46ec-aefa-692b6ae62721 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:35.652506 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 31/62] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:05:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:05:35.657848 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf427c54-85b6-4eb2-b3bf-98ffc4573809 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:35.658746 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf427c54-85b6-4eb2-b3bf-98ffc4573809 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:35.659316 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf427c54-85b6-4eb2-b3bf-98ffc4573809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:35.684721 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-cf427c54-85b6-4eb2-b3bf-98ffc4573809 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:05:35.686448 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cf427c54-85b6-4eb2-b3bf-98ffc4573809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:05:35.778527 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 32/63] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:05:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:05:35.814202 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1526a031-bcfc-46ec-aefa-692b6ae62721 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:35.852502 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:35.960841 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1526a031-bcfc-46ec-aefa-692b6ae62721 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:36.005917 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87dcffc-ed0e-4326-b414-91b9f5f94d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:36.007078 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87dcffc-ed0e-4326-b414-91b9f5f94d9d None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:36.007569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87dcffc-ed0e-4326-b414-91b9f5f94d9d None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:36.008052 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87dcffc-ed0e-4326-b414-91b9f5f94d9d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:36.025322 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87dcffc-ed0e-4326-b414-91b9f5f94d9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:05:36.033235 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-c87dcffc-ed0e-4326-b414-91b9f5f94d9d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:05:36.033235 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:36.033235 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:36.033235 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:36.033235 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:36.033235 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:36.033235 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:36.033235 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:36.033235 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:36.033235 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:36.033235 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:36.033235 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:36.033235 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:05:36.033235 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:05:36.033235 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:05:36.033235 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:05:36.034115 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:05:36.034115 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:05:36.034115 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:05:36.034115 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:05:36.034115 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:36.034115 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:36.034115 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:36.034115 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:36.034115 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:36.034115 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:36.034115 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:36.034115 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:36.034115 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:36.034115 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:36.034115 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:36.034115 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:36.034890 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:36.034890 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:36.034890 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:36.034890 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:36.034890 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:05:36.034890 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:05:36.034890 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:05:36.034890 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:05:36.034890 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:05:36.034890 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:05:36.034890 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:05:36.034890 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:05:36.034890 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 32/64] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:05:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:05:36.038595 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c1d051-a8dc-438c-bcb8-2224bdd74cd5 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:36.039181 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c1d051-a8dc-438c-bcb8-2224bdd74cd5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:36.039667 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c1d051-a8dc-438c-bcb8-2224bdd74cd5 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:36.040187 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87dcffc-ed0e-4326-b414-91b9f5f94d9d None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:36.041839 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 33/65] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:05:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:05:36.047231 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a682e151-5e3a-418a-8453-ed2880335c23 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:36.047770 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a682e151-5e3a-418a-8453-ed2880335c23 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:36.048261 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a682e151-5e3a-418a-8453-ed2880335c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:36.076991 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a682e151-5e3a-418a-8453-ed2880335c23 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:05:36.078575 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a682e151-5e3a-418a-8453-ed2880335c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:05:36.152140 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 34/66] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:05:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:05:36.157940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4209bcc8-c464-479f-8c5f-491d3179aa12 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:36.158468 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4209bcc8-c464-479f-8c5f-491d3179aa12 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:36.158980 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4209bcc8-c464-479f-8c5f-491d3179aa12 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:36.160607 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 35/67] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:05:36 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:05:36.172212 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:36.173484 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66861d2-a9ec-48b9-a597-d9caacfab0ab None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:36.173776 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66861d2-a9ec-48b9-a597-d9caacfab0ab None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:36.174064 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66861d2-a9ec-48b9-a597-d9caacfab0ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:36.185209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87dcffc-ed0e-4326-b414-91b9f5f94d9d None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:36.200187 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c66861d2-a9ec-48b9-a597-d9caacfab0ab None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:05:36.202029 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c66861d2-a9ec-48b9-a597-d9caacfab0ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:05:36.211680 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:36.226095 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536c70da-a4e8-4172-a6b8-177741bf75fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:36.227116 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c70da-a4e8-4172-a6b8-177741bf75fa None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:36.227608 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c70da-a4e8-4172-a6b8-177741bf75fa None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:36.228095 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c70da-a4e8-4172-a6b8-177741bf75fa None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:36.229182 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536c70da-a4e8-4172-a6b8-177741bf75fa None admin] RBAC: Adding query filter params (name=member) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:36.229725 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536c70da-a4e8-4172-a6b8-177741bf75fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:36.231461 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536c70da-a4e8-4172-a6b8-177741bf75fa None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:36.245451 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 33/68] 10.222.0.71 () {56 vars in 1101 bytes} [Mon Jul 4 09:05:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:05:36.246598 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536c70da-a4e8-4172-a6b8-177741bf75fa None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:36.283011 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 36/69] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:05:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:05:36.283824 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66861d2-a9ec-48b9-a597-d9caacfab0ab None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:36.372107 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536c70da-a4e8-4172-a6b8-177741bf75fa None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:36.388432 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66861d2-a9ec-48b9-a597-d9caacfab0ab None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:36.416286 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa0ec6-c616-48f9-a458-350be8b1a3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:36.417719 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa0ec6-c616-48f9-a458-350be8b1a3b8 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:36.418022 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa0ec6-c616-48f9-a458-350be8b1a3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:36.418291 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa0ec6-c616-48f9-a458-350be8b1a3b8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:36.426766 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49bb8ac-c695-42aa-86bc-99d52b8cb26f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:36.427650 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49bb8ac-c695-42aa-86bc-99d52b8cb26f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:36.427939 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49bb8ac-c695-42aa-86bc-99d52b8cb26f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:36.428222 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49bb8ac-c695-42aa-86bc-99d52b8cb26f None admin] PATH_INFO: `/v3/users/a8b858d3091e4e51b82bacac35f8daa7` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:36.429371 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aa0ec6-c616-48f9-a458-350be8b1a3b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:05:36.437643 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-b0aa0ec6-c616-48f9-a458-350be8b1a3b8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:05:36.437643 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:36.437643 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:36.437643 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:36.437643 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:36.437643 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:36.437643 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:36.437643 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:36.437643 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:36.437643 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:36.437643 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:36.437643 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:36.437643 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:05:36.437643 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:05:36.437643 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:05:36.437643 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:05:36.438524 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:05:36.438524 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:05:36.438524 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:05:36.438524 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:05:36.438524 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:36.438524 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:36.438524 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:36.438524 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:36.438524 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:36.438524 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:36.438524 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:36.438524 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:36.438524 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:36.438524 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:36.438524 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:36.438524 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:36.439286 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:36.439286 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:36.439286 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:36.439286 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:36.439286 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:05:36.439286 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:05:36.439286 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:05:36.439286 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:05:36.439286 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:05:36.439286 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:05:36.439286 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:05:36.439286 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:05:36.439286 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 34/70] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 4 09:05:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:05:36.440523 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68bc2fa-b649-4d0c-854b-404f102a34a2 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:36.440523 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68bc2fa-b649-4d0c-854b-404f102a34a2 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:36.440684 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68bc2fa-b649-4d0c-854b-404f102a34a2 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:36.445086 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 35/71] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:05:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:05:36.445086 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68bc2fa-b649-4d0c-854b-404f102a34a2 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:36.449947 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:36.453183 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49bb8ac-c695-42aa-86bc-99d52b8cb26f None admin] RBAC: Authorizing `identity:get_user(user_id=a8b858d3091e4e51b82bacac35f8daa7)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:36.454686 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49bb8ac-c695-42aa-86bc-99d52b8cb26f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:36.456560 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:36.456803 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 37/72] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 4 09:05:36 2022] GET /identity/v3/users/a8b858d3091e4e51b82bacac35f8daa7 => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:36.458057 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49bb8ac-c695-42aa-86bc-99d52b8cb26f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:36.555543 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68bc2fa-b649-4d0c-854b-404f102a34a2 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:36.559087 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49bb8ac-c695-42aa-86bc-99d52b8cb26f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:36.590243 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5a1a16-ca8f-486b-9585-f2a4e0034259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:36.591089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a1a16-ca8f-486b-9585-f2a4e0034259 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:36.591385 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a1a16-ca8f-486b-9585-f2a4e0034259 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:36.591675 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a1a16-ca8f-486b-9585-f2a4e0034259 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:36.592935 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f0a863-85f5-4056-9f06-ee9232b63123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:36.593631 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0a863-85f5-4056-9f06-ee9232b63123 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:36.593921 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0a863-85f5-4056-9f06-ee9232b63123 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:36.594203 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0a863-85f5-4056-9f06-ee9232b63123 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:36.594951 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f0a863-85f5-4056-9f06-ee9232b63123 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:36.595262 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f0a863-85f5-4056-9f06-ee9232b63123 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:36.596627 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f0a863-85f5-4056-9f06-ee9232b63123 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:36.601780 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5a1a16-ca8f-486b-9585-f2a4e0034259 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:05:36.605135 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 38/73] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:05:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:36.606388 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f0d24-3b67-4bc2-9bc3-9fe743985dc9 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:36.606686 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f0d24-3b67-4bc2-9bc3-9fe743985dc9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:36.606970 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f0d24-3b67-4bc2-9bc3-9fe743985dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:36.610815 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-7b5a1a16-ca8f-486b-9585-f2a4e0034259 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:05:36.610815 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:36.610815 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:36.610815 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:36.610815 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:36.610815 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:36.610815 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:36.610815 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:36.610815 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:36.610815 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:36.610815 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:36.610815 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:36.610815 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:05:36.610815 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:05:36.610815 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:05:36.610815 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:05:36.611774 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:05:36.611774 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:05:36.611774 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:05:36.611774 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:05:36.611774 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:36.611774 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:36.611774 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:36.611774 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:36.611774 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:36.611774 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:36.611774 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:36.611774 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:36.611774 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:36.611774 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:36.611774 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:36.611774 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:36.612599 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:36.612599 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:36.612599 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:36.612599 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:36.612599 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:05:36.612599 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:05:36.612599 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:05:36.612599 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:05:36.612599 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:05:36.612599 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:05:36.612599 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:05:36.612599 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:05:36.612599 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 36/74] 10.222.0.71 () {56 vars in 1114 bytes} [Mon Jul 4 09:05:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:05:36.613294 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5a1a16-ca8f-486b-9585-f2a4e0034259 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:36.628221 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f45f0d24-3b67-4bc2-9bc3-9fe743985dc9 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:05:36.629457 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f45f0d24-3b67-4bc2-9bc3-9fe743985dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:05:36.680119 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:36.685842 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:36.696550 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 39/75] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:05:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:05:36.698044 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac69c19-d853-4c33-b186-65a57524cdb9 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:36.698342 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac69c19-d853-4c33-b186-65a57524cdb9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:36.698736 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac69c19-d853-4c33-b186-65a57524cdb9 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:36.700166 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 40/76] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:05:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:05:36.700777 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac69c19-d853-4c33-b186-65a57524cdb9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:36.720770 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5a1a16-ca8f-486b-9585-f2a4e0034259 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:36.755372 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f35c73f-6e41-4b0c-80f8-e08c522076a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:36.756025 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f35c73f-6e41-4b0c-80f8-e08c522076a0 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:36.756312 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f35c73f-6e41-4b0c-80f8-e08c522076a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:36.756592 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f35c73f-6e41-4b0c-80f8-e08c522076a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:36.804445 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac69c19-d853-4c33-b186-65a57524cdb9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:36.824270 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f35c73f-6e41-4b0c-80f8-e08c522076a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:36.825740 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f35c73f-6e41-4b0c-80f8-e08c522076a0 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:36.839008 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feb1f39-9e0e-42cf-96fb-3fcacd537fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:36.840053 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb1f39-9e0e-42cf-96fb-3fcacd537fd5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:36.840342 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb1f39-9e0e-42cf-96fb-3fcacd537fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:36.840625 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb1f39-9e0e-42cf-96fb-3fcacd537fd5 None admin] PATH_INFO: `/v3/users/fa43c0e65110476d9e52a93b45c1a988` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:36.841858 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feb1f39-9e0e-42cf-96fb-3fcacd537fd5 None admin] RBAC: Authorizing `identity:get_user(user_id=fa43c0e65110476d9e52a93b45c1a988)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:36.843298 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feb1f39-9e0e-42cf-96fb-3fcacd537fd5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:36.844921 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 41/77] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 4 09:05:36 2022] GET /identity/v3/users/fa43c0e65110476d9e52a93b45c1a988 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:36.845737 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feb1f39-9e0e-42cf-96fb-3fcacd537fd5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:36.891963 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:36.907080 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:36.929185 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 37/78] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:05:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:05:36.930459 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61ec139-3f48-44e2-9c06-5b176a49df15 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:36.930753 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61ec139-3f48-44e2-9c06-5b176a49df15 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:36.931033 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61ec139-3f48-44e2-9c06-5b176a49df15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:36.947152 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-b61ec139-3f48-44e2-9c06-5b176a49df15 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:05:36.949504 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b61ec139-3f48-44e2-9c06-5b176a49df15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:05:36.954733 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feb1f39-9e0e-42cf-96fb-3fcacd537fd5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:36.990066 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e5362e-ccc8-442b-a47d-6ae367f97c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:36.990732 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e5362e-ccc8-442b-a47d-6ae367f97c20 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:36.991020 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e5362e-ccc8-442b-a47d-6ae367f97c20 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:36.991303 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e5362e-ccc8-442b-a47d-6ae367f97c20 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:36.992047 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e5362e-ccc8-442b-a47d-6ae367f97c20 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:36.992361 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e5362e-ccc8-442b-a47d-6ae367f97c20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:36.993771 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e5362e-ccc8-442b-a47d-6ae367f97c20 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:37.001920 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 42/79] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 4 09:05:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:37.002708 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e5362e-ccc8-442b-a47d-6ae367f97c20 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:37.019895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 38/80] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:05:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:05:37.020739 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61ec139-3f48-44e2-9c06-5b176a49df15 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:37.115249 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e5362e-ccc8-442b-a47d-6ae367f97c20 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:37.117085 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:37.121375 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61ec139-3f48-44e2-9c06-5b176a49df15 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:37.137090 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:37.148044 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7f59be-4b8c-41b4-924d-64aa18a96a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:37.148841 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f59be-4b8c-41b4-924d-64aa18a96a19 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:37.149156 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f59be-4b8c-41b4-924d-64aa18a96a19 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:37.149440 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f59be-4b8c-41b4-924d-64aa18a96a19 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:37.150612 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7f59be-4b8c-41b4-924d-64aa18a96a19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:37.152760 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7f59be-4b8c-41b4-924d-64aa18a96a19 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:37.154415 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 43/81] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:05:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:37.155172 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7f59be-4b8c-41b4-924d-64aa18a96a19 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:37.156259 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d43a3-6663-4d83-a095-4d4076cf29c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:37.156915 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d43a3-6663-4d83-a095-4d4076cf29c4 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:37.157245 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d43a3-6663-4d83-a095-4d4076cf29c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:37.157527 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d43a3-6663-4d83-a095-4d4076cf29c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:37.225448 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111d43a3-6663-4d83-a095-4d4076cf29c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:37.226945 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111d43a3-6663-4d83-a095-4d4076cf29c4 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:37.252873 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7f59be-4b8c-41b4-924d-64aa18a96a19 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:37.287849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76999063-b6e5-4be4-b67d-0f55fcf69d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:37.288705 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76999063-b6e5-4be4-b67d-0f55fcf69d7b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:37.288992 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76999063-b6e5-4be4-b67d-0f55fcf69d7b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:37.289300 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76999063-b6e5-4be4-b67d-0f55fcf69d7b None admin] PATH_INFO: `/v3/projects/e192ac889bad428985088eea5ce750ab` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:37.290553 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76999063-b6e5-4be4-b67d-0f55fcf69d7b None admin] RBAC: Authorizing `identity:get_project(project_id=e192ac889bad428985088eea5ce750ab)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:37.292493 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76999063-b6e5-4be4-b67d-0f55fcf69d7b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:37.294223 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 44/82] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 4 09:05:37 2022] GET /identity/v3/projects/e192ac889bad428985088eea5ce750ab => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:37.294986 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76999063-b6e5-4be4-b67d-0f55fcf69d7b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:37.321989 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:37.328676 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 39/83] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:05:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:05:37.334043 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d43a3-6663-4d83-a095-4d4076cf29c4 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:37.360860 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:37.405852 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76999063-b6e5-4be4-b67d-0f55fcf69d7b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:37.438494 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d43a3-6663-4d83-a095-4d4076cf29c4 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:37.439357 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45326eb6-7ef3-48a6-a37f-8a3647832afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:37.440248 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45326eb6-7ef3-48a6-a37f-8a3647832afb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:37.440550 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45326eb6-7ef3-48a6-a37f-8a3647832afb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:37.440836 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45326eb6-7ef3-48a6-a37f-8a3647832afb None admin] PATH_INFO: `/v3/users/a8b858d3091e4e51b82bacac35f8daa7` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:37.442078 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45326eb6-7ef3-48a6-a37f-8a3647832afb None admin] RBAC: Authorizing `identity:get_user(user_id=a8b858d3091e4e51b82bacac35f8daa7)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:37.443526 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45326eb6-7ef3-48a6-a37f-8a3647832afb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:37.445186 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 45/84] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 4 09:05:37 2022] GET /identity/v3/users/a8b858d3091e4e51b82bacac35f8daa7 => generated 275 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:37.445942 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45326eb6-7ef3-48a6-a37f-8a3647832afb None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:37.474094 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a074b9c-6952-4b34-84ed-f279bb45bedd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:37.474902 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a074b9c-6952-4b34-84ed-f279bb45bedd None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:37.475197 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a074b9c-6952-4b34-84ed-f279bb45bedd None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:37.475485 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a074b9c-6952-4b34-84ed-f279bb45bedd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:37.486439 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a074b9c-6952-4b34-84ed-f279bb45bedd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:05:37.494203 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-5a074b9c-6952-4b34-84ed-f279bb45bedd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:05:37.494203 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:37.494203 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:37.494203 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:37.494203 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:37.494203 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:37.494203 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:37.494203 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:37.494203 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:37.494203 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:37.494203 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:37.494203 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:37.494203 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:05:37.494203 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:05:37.494203 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:05:37.494203 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:05:37.495798 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:05:37.495798 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:05:37.495798 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:05:37.495798 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:05:37.495798 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:37.495798 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:37.495798 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:37.495798 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:37.495798 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:37.495798 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:37.495798 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:37.495798 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:37.495798 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:37.495798 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:37.495798 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:37.495798 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:37.496611 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:37.496611 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:37.496611 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:37.496611 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:37.496611 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:05:37.496611 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:05:37.496611 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:05:37.496611 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:05:37.496611 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:05:37.496611 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:05:37.496611 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:05:37.496611 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:05:37.496611 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 40/85] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:05:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:05:37.497840 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a074b9c-6952-4b34-84ed-f279bb45bedd None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:37.548030 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45326eb6-7ef3-48a6-a37f-8a3647832afb None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:37.554247 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:37.575362 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:37.583942 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bba5f0-ccf8-4cb4-952d-31cb226da64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:37.584619 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bba5f0-ccf8-4cb4-952d-31cb226da64f None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:37.584995 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bba5f0-ccf8-4cb4-952d-31cb226da64f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:37.585336 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bba5f0-ccf8-4cb4-952d-31cb226da64f None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:37.586064 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bba5f0-ccf8-4cb4-952d-31cb226da64f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:37.587758 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bba5f0-ccf8-4cb4-952d-31cb226da64f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:37.597202 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a074b9c-6952-4b34-84ed-f279bb45bedd None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:37.607826 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 46/86] 10.222.0.71 () {60 vars in 1132 bytes} [Mon Jul 4 09:05:37 2022] POST /identity/v3/users => generated 289 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:05:37.608642 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bba5f0-ccf8-4cb4-952d-31cb226da64f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:37.636689 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9caf73a-0e04-4b25-b365-cfe87ad59eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:37.637411 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9caf73a-0e04-4b25-b365-cfe87ad59eef None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:37.637705 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9caf73a-0e04-4b25-b365-cfe87ad59eef None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:37.637988 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9caf73a-0e04-4b25-b365-cfe87ad59eef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:37.638851 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9caf73a-0e04-4b25-b365-cfe87ad59eef None admin] RBAC: Adding query filter params (role.id=8692e99e9463459f9b7caa96b3051b93, scope.project.id=e192ac889bad428985088eea5ce750ab, user.id=a8b858d3091e4e51b82bacac35f8daa7) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:37.639176 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9caf73a-0e04-4b25-b365-cfe87ad59eef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:37.640954 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9caf73a-0e04-4b25-b365-cfe87ad59eef None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:37.661200 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 41/87] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 4 09:05:37 2022] GET /identity/v3/role_assignments?user.id=a8b858d3091e4e51b82bacac35f8daa7&scope.project.id=e192ac889bad428985088eea5ce750ab&role.id=8692e99e9463459f9b7caa96b3051b93 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:37.661200 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9caf73a-0e04-4b25-b365-cfe87ad59eef None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:37.748131 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bba5f0-ccf8-4cb4-952d-31cb226da64f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:37.796077 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b41b6af-e1f6-43b8-9efc-88b07942d6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:37.797165 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b41b6af-e1f6-43b8-9efc-88b07942d6c7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:37.797460 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b41b6af-e1f6-43b8-9efc-88b07942d6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:37.797750 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b41b6af-e1f6-43b8-9efc-88b07942d6c7 None admin] PATH_INFO: `/v3/projects/e192ac889bad428985088eea5ce750ab` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:37.799046 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b41b6af-e1f6-43b8-9efc-88b07942d6c7 None admin] RBAC: Authorizing `identity:get_project(project_id=e192ac889bad428985088eea5ce750ab)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:37.800555 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b41b6af-e1f6-43b8-9efc-88b07942d6c7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:37.802401 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 47/88] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 4 09:05:37 2022] GET /identity/v3/projects/e192ac889bad428985088eea5ce750ab => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:37.803275 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b41b6af-e1f6-43b8-9efc-88b07942d6c7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:37.818661 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9caf73a-0e04-4b25-b365-cfe87ad59eef None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:37.849724 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:37.863930 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243c3008-b80e-49d1-b2b1-9a35c530d2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:37.864599 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243c3008-b80e-49d1-b2b1-9a35c530d2c8 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:37.864892 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243c3008-b80e-49d1-b2b1-9a35c530d2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:37.865229 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243c3008-b80e-49d1-b2b1-9a35c530d2c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:37.865327 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:37.933102 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b41b6af-e1f6-43b8-9efc-88b07942d6c7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:37.946593 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243c3008-b80e-49d1-b2b1-9a35c530d2c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:37.948676 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243c3008-b80e-49d1-b2b1-9a35c530d2c8 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:37.976351 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04cab2-3b96-40b3-94cb-4d4f301c98c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:37.977324 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04cab2-3b96-40b3-94cb-4d4f301c98c8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:37.977784 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04cab2-3b96-40b3-94cb-4d4f301c98c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:37.978259 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04cab2-3b96-40b3-94cb-4d4f301c98c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:37.979062 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a04cab2-3b96-40b3-94cb-4d4f301c98c8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:37.979440 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a04cab2-3b96-40b3-94cb-4d4f301c98c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:37.980847 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a04cab2-3b96-40b3-94cb-4d4f301c98c8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:37.991027 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 48/89] 10.222.0.71 () {56 vars in 1101 bytes} [Mon Jul 4 09:05:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:37.991877 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04cab2-3b96-40b3-94cb-4d4f301c98c8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:38.069416 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 42/90] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:05:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:05:38.070471 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243c3008-b80e-49d1-b2b1-9a35c530d2c8 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:38.112187 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:38.141913 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04cab2-3b96-40b3-94cb-4d4f301c98c8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:38.161186 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:38.214409 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191fa411-fe5c-495e-8486-42ab924e479a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:38.215080 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191fa411-fe5c-495e-8486-42ab924e479a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:38.215372 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191fa411-fe5c-495e-8486-42ab924e479a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:38.215659 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191fa411-fe5c-495e-8486-42ab924e479a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:38.216525 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191fa411-fe5c-495e-8486-42ab924e479a None admin] RBAC: Adding query filter params (role.id=f36e053b442540f08041ac7878760f65, scope.project.id=e192ac889bad428985088eea5ce750ab, user.id=fa43c0e65110476d9e52a93b45c1a988) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:38.216844 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191fa411-fe5c-495e-8486-42ab924e479a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:38.218331 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191fa411-fe5c-495e-8486-42ab924e479a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:38.222514 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243c3008-b80e-49d1-b2b1-9a35c530d2c8 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:38.238837 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 49/91] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 4 09:05:37 2022] GET /identity/v3/role_assignments?user.id=fa43c0e65110476d9e52a93b45c1a988&scope.project.id=e192ac889bad428985088eea5ce750ab&role.id=f36e053b442540f08041ac7878760f65 => generated 259 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:38.239716 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191fa411-fe5c-495e-8486-42ab924e479a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:38.262776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214ebf45-19d4-4fa9-8e8f-21e98fde8288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:38.264058 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ebf45-19d4-4fa9-8e8f-21e98fde8288 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:38.264485 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ebf45-19d4-4fa9-8e8f-21e98fde8288 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:38.264948 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ebf45-19d4-4fa9-8e8f-21e98fde8288 None admin] PATH_INFO: `/v3/users/a8b858d3091e4e51b82bacac35f8daa7` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:38.266485 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214ebf45-19d4-4fa9-8e8f-21e98fde8288 None admin] RBAC: Authorizing `identity:get_user(user_id=a8b858d3091e4e51b82bacac35f8daa7)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:38.268174 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214ebf45-19d4-4fa9-8e8f-21e98fde8288 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:38.270324 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 43/92] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 4 09:05:38 2022] GET /identity/v3/users/a8b858d3091e4e51b82bacac35f8daa7 => generated 275 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:38.276824 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214ebf45-19d4-4fa9-8e8f-21e98fde8288 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:38.457617 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191fa411-fe5c-495e-8486-42ab924e479a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:38.477230 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:38.499873 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214ebf45-19d4-4fa9-8e8f-21e98fde8288 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:38.524182 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea58652b-ce30-4ec2-977c-db8fda70ff9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:38.525348 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea58652b-ce30-4ec2-977c-db8fda70ff9a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:38.525837 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea58652b-ce30-4ec2-977c-db8fda70ff9a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:38.526951 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea58652b-ce30-4ec2-977c-db8fda70ff9a None admin] PATH_INFO: `/v3/projects/e192ac889bad428985088eea5ce750ab` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:38.528374 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea58652b-ce30-4ec2-977c-db8fda70ff9a None admin] RBAC: Authorizing `identity:get_project(project_id=e192ac889bad428985088eea5ce750ab)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:38.530023 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea58652b-ce30-4ec2-977c-db8fda70ff9a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:38.531889 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 50/93] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 4 09:05:38 2022] GET /identity/v3/projects/e192ac889bad428985088eea5ce750ab => generated 298 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:38.537553 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea58652b-ce30-4ec2-977c-db8fda70ff9a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:38.573780 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc3ad5d-3deb-463c-a2a3-42e098dd4543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:38.574456 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc3ad5d-3deb-463c-a2a3-42e098dd4543 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:38.574747 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc3ad5d-3deb-463c-a2a3-42e098dd4543 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:38.575036 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc3ad5d-3deb-463c-a2a3-42e098dd4543 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:38.599870 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:38.713997 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea58652b-ce30-4ec2-977c-db8fda70ff9a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:38.733124 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc3ad5d-3deb-463c-a2a3-42e098dd4543 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:38.738690 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc3ad5d-3deb-463c-a2a3-42e098dd4543 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:38.810312 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c07e2a-401d-4292-b005-c84efd3e8fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:38.810988 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c07e2a-401d-4292-b005-c84efd3e8fa7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:38.811282 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c07e2a-401d-4292-b005-c84efd3e8fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:38.811569 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c07e2a-401d-4292-b005-c84efd3e8fa7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:38.812416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c07e2a-401d-4292-b005-c84efd3e8fa7 None admin] RBAC: Adding query filter params (role.id=5fb58444d5604bf689014e27dccacb45, scope.project.id=e192ac889bad428985088eea5ce750ab, user.id=a8b858d3091e4e51b82bacac35f8daa7) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:38.812736 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c07e2a-401d-4292-b005-c84efd3e8fa7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:38.814222 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c07e2a-401d-4292-b005-c84efd3e8fa7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:38.831172 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 51/94] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 4 09:05:38 2022] GET /identity/v3/role_assignments?user.id=a8b858d3091e4e51b82bacac35f8daa7&scope.project.id=e192ac889bad428985088eea5ce750ab&role.id=5fb58444d5604bf689014e27dccacb45 => generated 259 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:05:38.913054 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:38.934081 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 44/95] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:05:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 658 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:05:38.939923 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c07e2a-401d-4292-b005-c84efd3e8fa7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:39.083650 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c07e2a-401d-4292-b005-c84efd3e8fa7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:39.103209 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:39.132151 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758146c9-a96a-42bf-97fd-124e48c0e9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:39.133061 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758146c9-a96a-42bf-97fd-124e48c0e9c6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:39.133355 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758146c9-a96a-42bf-97fd-124e48c0e9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:39.133640 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758146c9-a96a-42bf-97fd-124e48c0e9c6 None admin] PATH_INFO: `/v3/projects/de987c0b1cd242759308014bda6eb4bd` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:39.137878 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758146c9-a96a-42bf-97fd-124e48c0e9c6 None admin] RBAC: Authorizing `identity:get_project(project_id=de987c0b1cd242759308014bda6eb4bd)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:39.139362 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758146c9-a96a-42bf-97fd-124e48c0e9c6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:39.141681 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 52/96] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 4 09:05:38 2022] GET /identity/v3/projects/de987c0b1cd242759308014bda6eb4bd => generated 312 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:39.147514 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc3ad5d-3deb-463c-a2a3-42e098dd4543 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:39.300097 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc3ad5d-3deb-463c-a2a3-42e098dd4543 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:39.354232 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b57037-279c-4245-9b1d-e2848aca0e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:39.354676 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b57037-279c-4245-9b1d-e2848aca0e91 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:39.354967 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b57037-279c-4245-9b1d-e2848aca0e91 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:39.355253 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b57037-279c-4245-9b1d-e2848aca0e91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:39.356181 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b57037-279c-4245-9b1d-e2848aca0e91 None admin] RBAC: Adding query filter params (role.id=8692e99e9463459f9b7caa96b3051b93, scope.project.id=de987c0b1cd242759308014bda6eb4bd, user.id=a8b858d3091e4e51b82bacac35f8daa7) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:39.356503 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b57037-279c-4245-9b1d-e2848aca0e91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:39.358019 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b57037-279c-4245-9b1d-e2848aca0e91 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:39.369324 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 45/97] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 4 09:05:39 2022] GET /identity/v3/role_assignments?user.id=a8b858d3091e4e51b82bacac35f8daa7&scope.project.id=de987c0b1cd242759308014bda6eb4bd&role.id=8692e99e9463459f9b7caa96b3051b93 => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:40.498100 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8966ad25-2bcb-4ecf-ad1d-06f634ea0ca3 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:40.498601 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8966ad25-2bcb-4ecf-ad1d-06f634ea0ca3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:40.498673 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8966ad25-2bcb-4ecf-ad1d-06f634ea0ca3 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:40.500315 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 53/98] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:05:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:05:40.505701 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9237d2-0c4a-441a-8922-ed4a4b56e0f0 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:40.506090 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9237d2-0c4a-441a-8922-ed4a4b56e0f0 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:40.506273 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9237d2-0c4a-441a-8922-ed4a4b56e0f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:40.529063 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-9b9237d2-0c4a-441a-8922-ed4a4b56e0f0 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:05:40.530653 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-9b9237d2-0c4a-441a-8922-ed4a4b56e0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:05:40.558275 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d876e-c142-4165-9b68-408b142a230e None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:40.558589 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d876e-c142-4165-9b68-408b142a230e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:40.558850 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d876e-c142-4165-9b68-408b142a230e None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:40.560514 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 54/99] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:05:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:05:40.565081 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c749d66-6d48-4ede-95fc-67352830a8c4 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:40.565381 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c749d66-6d48-4ede-95fc-67352830a8c4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:40.565666 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c749d66-6d48-4ede-95fc-67352830a8c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:40.594676 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1c749d66-6d48-4ede-95fc-67352830a8c4 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:05:40.595266 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1c749d66-6d48-4ede-95fc-67352830a8c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:05:40.626876 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:40.660045 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 46/100] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:05:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:05:40.706476 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9237d2-0c4a-441a-8922-ed4a4b56e0f0 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:40.720925 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 55/101] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:05:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:05:40.807654 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c749d66-6d48-4ede-95fc-67352830a8c4 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:40.861878 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9237d2-0c4a-441a-8922-ed4a4b56e0f0 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:40.900436 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf30aca-0487-42bc-a181-14c33a21df15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:40.901920 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf30aca-0487-42bc-a181-14c33a21df15 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:40.902212 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf30aca-0487-42bc-a181-14c33a21df15 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:40.902495 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf30aca-0487-42bc-a181-14c33a21df15 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:40.914539 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf30aca-0487-42bc-a181-14c33a21df15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:05:40.922296 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-0bf30aca-0487-42bc-a181-14c33a21df15 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 09:05:40.922296 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:40.922296 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:40.922296 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:40.922296 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:40.922296 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:40.922296 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:40.922296 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:40.922296 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:40.922296 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:40.922296 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:40.922296 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:40.922296 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:05:40.922296 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:05:40.922296 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:05:40.922296 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:05:40.923198 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:05:40.923198 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:05:40.923198 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:05:40.923198 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:05:40.923198 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:40.923198 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:40.923198 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:40.923198 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:40.923198 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:40.923198 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:40.923198 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:40.923198 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:40.923198 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:40.923198 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:40.923198 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:40.923198 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:40.923974 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:40.923974 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:40.923974 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:40.923974 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:40.923974 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:05:40.923974 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:05:40.923974 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:05:40.923974 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:05:40.923974 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:05:40.923974 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:05:40.923974 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 09:05:40.923974 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:05:40.923974 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 47/102] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:05:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:05:40.925128 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31dfe68-ef38-4e93-9c3c-1a45ab0010d9 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:40.925128 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31dfe68-ef38-4e93-9c3c-1a45ab0010d9 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:40.925406 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31dfe68-ef38-4e93-9c3c-1a45ab0010d9 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:40.926935 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 48/103] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:05:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:05:40.931627 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31dfe68-ef38-4e93-9c3c-1a45ab0010d9 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:40.941408 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c749d66-6d48-4ede-95fc-67352830a8c4 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:40.950958 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:40.982272 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace6331f-ce9d-4113-a1da-6296db2ba57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:40.983371 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace6331f-ce9d-4113-a1da-6296db2ba57d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:40.983633 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:40.984415 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace6331f-ce9d-4113-a1da-6296db2ba57d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:40.984886 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace6331f-ce9d-4113-a1da-6296db2ba57d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:40.997584 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace6331f-ce9d-4113-a1da-6296db2ba57d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:05:41.006399 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-ace6331f-ce9d-4113-a1da-6296db2ba57d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:05:41.006399 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:41.006399 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:41.006399 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:41.006399 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:41.006399 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:41.006399 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:41.006399 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:41.006399 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:41.006399 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:41.006399 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:41.006399 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:41.006399 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:05:41.006399 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:05:41.006399 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:05:41.006399 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:05:41.007256 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:05:41.007256 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:05:41.007256 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:05:41.007256 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:05:41.007256 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:41.007256 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:41.007256 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:41.007256 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:41.007256 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:41.007256 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:41.007256 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:41.007256 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:41.007256 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:41.007256 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:41.007256 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:41.007256 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:41.008034 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:41.008034 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:41.008034 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:41.008034 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:41.008034 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:05:41.008034 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:05:41.008034 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:05:41.008034 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:05:41.008034 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:05:41.008034 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:05:41.008034 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:05:41.008034 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:05:41.011859 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 56/104] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:05:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:05:41.017408 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e6932-720a-4874-aacd-efa55511944b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:41.017938 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e6932-720a-4874-aacd-efa55511944b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:41.018494 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e6932-720a-4874-aacd-efa55511944b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:41.037899 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b51e6932-720a-4874-aacd-efa55511944b None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:05:41.039593 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b51e6932-720a-4874-aacd-efa55511944b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:05:41.044017 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31dfe68-ef38-4e93-9c3c-1a45ab0010d9 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:41.079543 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f8efbe-342c-46a6-abc7-04e8af476702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:41.080236 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f8efbe-342c-46a6-abc7-04e8af476702 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:41.080528 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f8efbe-342c-46a6-abc7-04e8af476702 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:41.080841 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f8efbe-342c-46a6-abc7-04e8af476702 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:41.081673 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f8efbe-342c-46a6-abc7-04e8af476702 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:41.081992 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f8efbe-342c-46a6-abc7-04e8af476702 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:41.084157 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f8efbe-342c-46a6-abc7-04e8af476702 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:41.093971 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 49/105] 10.222.0.71 () {56 vars in 1101 bytes} [Mon Jul 4 09:05:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:41.094805 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f8efbe-342c-46a6-abc7-04e8af476702 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:41.130553 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 57/106] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:05:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:05:41.131910 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e6932-720a-4874-aacd-efa55511944b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:41.238311 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:41.246099 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e6932-720a-4874-aacd-efa55511944b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:41.284220 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7071f3cc-446a-4575-a89e-47e9270932e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:41.285583 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7071f3cc-446a-4575-a89e-47e9270932e7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:41.286177 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7071f3cc-446a-4575-a89e-47e9270932e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:41.286660 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7071f3cc-446a-4575-a89e-47e9270932e7 None admin] PATH_INFO: `/v3/users/639cd4e0d5ca4e4d8f6bab272d48a607` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:41.301678 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7071f3cc-446a-4575-a89e-47e9270932e7 None admin] RBAC: Authorizing `identity:get_user(user_id=639cd4e0d5ca4e4d8f6bab272d48a607)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:41.303517 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7071f3cc-446a-4575-a89e-47e9270932e7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:41.305799 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 58/107] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 4 09:05:41 2022] GET /identity/v3/users/639cd4e0d5ca4e4d8f6bab272d48a607 => generated 289 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:41.306775 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7071f3cc-446a-4575-a89e-47e9270932e7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:41.335878 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f8efbe-342c-46a6-abc7-04e8af476702 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:41.347467 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:41.387963 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e6be5f-00db-4dbc-bb1f-a2f66cac03be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:41.391376 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6be5f-00db-4dbc-bb1f-a2f66cac03be None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:41.391376 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6be5f-00db-4dbc-bb1f-a2f66cac03be None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:41.391376 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6be5f-00db-4dbc-bb1f-a2f66cac03be None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:41.391376 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e6be5f-00db-4dbc-bb1f-a2f66cac03be None admin] RBAC: Adding query filter params (name=member) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:41.391376 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e6be5f-00db-4dbc-bb1f-a2f66cac03be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:41.397655 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e6be5f-00db-4dbc-bb1f-a2f66cac03be None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:41.404606 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 50/108] 10.222.0.71 () {56 vars in 1101 bytes} [Mon Jul 4 09:05:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:41.405613 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e6be5f-00db-4dbc-bb1f-a2f66cac03be None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:41.470902 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7071f3cc-446a-4575-a89e-47e9270932e7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:41.507944 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eb93ba-a4da-4e48-82e5-e7424eece383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:41.509431 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eb93ba-a4da-4e48-82e5-e7424eece383 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:41.509858 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eb93ba-a4da-4e48-82e5-e7424eece383 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:41.511281 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e6be5f-00db-4dbc-bb1f-a2f66cac03be None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:41.511764 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eb93ba-a4da-4e48-82e5-e7424eece383 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:41.513170 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eb93ba-a4da-4e48-82e5-e7424eece383 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:41.514817 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eb93ba-a4da-4e48-82e5-e7424eece383 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:41.516588 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 59/109] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:05:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:41.521971 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eb93ba-a4da-4e48-82e5-e7424eece383 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:41.546916 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f00467-4cd4-4c16-b617-c42f6e439756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:41.547588 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f00467-4cd4-4c16-b617-c42f6e439756 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:41.547889 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f00467-4cd4-4c16-b617-c42f6e439756 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:41.548161 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f00467-4cd4-4c16-b617-c42f6e439756 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:41.557377 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:41.585505 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:41.621425 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f00467-4cd4-4c16-b617-c42f6e439756 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:41.623135 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f00467-4cd4-4c16-b617-c42f6e439756 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:41.640396 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eb93ba-a4da-4e48-82e5-e7424eece383 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:41.677758 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec768da8-54e1-4217-bf5f-d71c67948dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:41.678820 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec768da8-54e1-4217-bf5f-d71c67948dee None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:41.679313 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec768da8-54e1-4217-bf5f-d71c67948dee None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:41.679708 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec768da8-54e1-4217-bf5f-d71c67948dee None admin] PATH_INFO: `/v3/users/a8b858d3091e4e51b82bacac35f8daa7` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:41.681189 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec768da8-54e1-4217-bf5f-d71c67948dee None admin] RBAC: Authorizing `identity:get_user(user_id=a8b858d3091e4e51b82bacac35f8daa7)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:41.683012 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec768da8-54e1-4217-bf5f-d71c67948dee None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:41.689724 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 60/110] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 4 09:05:41 2022] GET /identity/v3/users/a8b858d3091e4e51b82bacac35f8daa7 => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:41.691242 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccfa6a0-6e28-45b0-a074-f26dab8ab74f None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:41.691989 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccfa6a0-6e28-45b0-a074-f26dab8ab74f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:41.692107 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccfa6a0-6e28-45b0-a074-f26dab8ab74f None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:41.693735 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 61/111] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:05:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:05:41.694544 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccfa6a0-6e28-45b0-a074-f26dab8ab74f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:41.731126 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 51/112] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:05:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:05:41.732216 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f00467-4cd4-4c16-b617-c42f6e439756 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:41.801026 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:41.808554 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:41.818981 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccfa6a0-6e28-45b0-a074-f26dab8ab74f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:41.838324 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f00467-4cd4-4c16-b617-c42f6e439756 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:41.858115 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d03f999-d2ca-413b-a2f4-b548959eb5ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:41.858794 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d03f999-d2ca-413b-a2f4-b548959eb5ec None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:41.859088 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d03f999-d2ca-413b-a2f4-b548959eb5ec None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:41.859375 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d03f999-d2ca-413b-a2f4-b548959eb5ec None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:41.860123 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d03f999-d2ca-413b-a2f4-b548959eb5ec None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:41.861727 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d03f999-d2ca-413b-a2f4-b548959eb5ec None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:41.876410 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00963ce-50ab-47d0-87e8-869cbac793a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:41.878301 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00963ce-50ab-47d0-87e8-869cbac793a6 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:41.878743 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00963ce-50ab-47d0-87e8-869cbac793a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:41.879223 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00963ce-50ab-47d0-87e8-869cbac793a6 None admin] PATH_INFO: `/v3/projects/e192ac889bad428985088eea5ce750ab` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:41.880958 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00963ce-50ab-47d0-87e8-869cbac793a6 None admin] RBAC: Authorizing `identity:get_project(project_id=e192ac889bad428985088eea5ce750ab)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:41.882733 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00963ce-50ab-47d0-87e8-869cbac793a6 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:41.885731 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 52/113] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 4 09:05:41 2022] GET /identity/v3/projects/e192ac889bad428985088eea5ce750ab => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:41.887469 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a845f34-2b4a-4ef8-af95-0280f8f83a33 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:41.887965 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a845f34-2b4a-4ef8-af95-0280f8f83a33 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:41.888471 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a845f34-2b4a-4ef8-af95-0280f8f83a33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:41.892518 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 62/114] 10.222.0.71 () {60 vars in 1141 bytes} [Mon Jul 4 09:05:41 2022] POST /identity/v3/projects => generated 302 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:05:41.901891 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d03f999-d2ca-413b-a2f4-b548959eb5ec None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:41.908069 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-8a845f34-2b4a-4ef8-af95-0280f8f83a33 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:05:41.909668 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-8a845f34-2b4a-4ef8-af95-0280f8f83a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:05:41.993814 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 53/115] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:05:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:05:41.995314 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a845f34-2b4a-4ef8-af95-0280f8f83a33 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:42.081484 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d03f999-d2ca-413b-a2f4-b548959eb5ec None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:42.106206 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:42.127369 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d95b32f-8086-4dd3-9466-1471472831d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:42.128825 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d95b32f-8086-4dd3-9466-1471472831d0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:42.129218 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d95b32f-8086-4dd3-9466-1471472831d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:42.129787 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d95b32f-8086-4dd3-9466-1471472831d0 None admin] PATH_INFO: `/v3/projects/e192ac889bad428985088eea5ce750ab` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:42.131196 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d95b32f-8086-4dd3-9466-1471472831d0 None admin] RBAC: Authorizing `identity:get_project(project_id=e192ac889bad428985088eea5ce750ab)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:42.132981 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d95b32f-8086-4dd3-9466-1471472831d0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:42.136123 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 63/116] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 4 09:05:41 2022] GET /identity/v3/projects/e192ac889bad428985088eea5ce750ab => generated 298 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:42.137560 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d880906c-d810-47ad-a562-9cef61be1a50 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:42.138115 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d880906c-d810-47ad-a562-9cef61be1a50 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:42.138414 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d880906c-d810-47ad-a562-9cef61be1a50 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:42.140294 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 64/117] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:05:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:05:42.141246 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d880906c-d810-47ad-a562-9cef61be1a50 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:42.151901 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a845f34-2b4a-4ef8-af95-0280f8f83a33 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:42.159960 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:42.195718 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c95e3a3-3460-4262-aaaa-3685a45d9f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:42.196753 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95e3a3-3460-4262-aaaa-3685a45d9f27 None admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:42.197160 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95e3a3-3460-4262-aaaa-3685a45d9f27 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:42.198081 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95e3a3-3460-4262-aaaa-3685a45d9f27 None admin] PATH_INFO: `/v3/projects/e192ac889bad428985088eea5ce750ab/users/a8b858d3091e4e51b82bacac35f8daa7/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:42.200422 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c95e3a3-3460-4262-aaaa-3685a45d9f27 None admin] RBAC: Authorizing `identity:create_grant(project_id=e192ac889bad428985088eea5ce750ab, user_id=a8b858d3091e4e51b82bacac35f8daa7, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:42.202046 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c95e3a3-3460-4262-aaaa-3685a45d9f27 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:42.222287 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 54/118] 10.222.0.71 () {58 vars in 1439 bytes} [Mon Jul 4 09:05:41 2022] PUT /identity/v3/projects/e192ac889bad428985088eea5ce750ab/users/a8b858d3091e4e51b82bacac35f8daa7/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:05:42.223015 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c95e3a3-3460-4262-aaaa-3685a45d9f27 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:42.280682 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d880906c-d810-47ad-a562-9cef61be1a50 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:42.327568 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6859601-bbab-4f36-9447-a6f66bfe1b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:42.328887 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6859601-bbab-4f36-9447-a6f66bfe1b5f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:42.329395 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6859601-bbab-4f36-9447-a6f66bfe1b5f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:42.329958 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6859601-bbab-4f36-9447-a6f66bfe1b5f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:42.345205 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6859601-bbab-4f36-9447-a6f66bfe1b5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:05:42.355185 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-d6859601-bbab-4f36-9447-a6f66bfe1b5f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:05:42.355185 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:42.355185 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:42.355185 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:42.355185 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:42.355185 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:42.355185 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:42.355185 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:42.355185 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:42.355185 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:42.355185 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:42.355185 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:42.355185 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:05:42.355185 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:05:42.355185 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:05:42.355185 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:05:42.356140 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:05:42.356140 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:05:42.356140 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:05:42.356140 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:05:42.356140 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:42.356140 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:42.356140 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:42.356140 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:42.356140 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:42.356140 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:42.356140 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:42.356140 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:42.356140 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:42.356140 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:42.356140 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:42.356140 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:42.357442 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:42.357442 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:42.357442 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:42.357442 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:42.357442 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:05:42.357442 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:05:42.357442 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:05:42.357442 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:05:42.357442 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:05:42.357442 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:05:42.357442 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:05:42.357442 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:05:42.357442 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 65/119] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 4 09:05:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:05:42.358262 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71073f8e-94d8-4013-95a2-fa72af5f20ed None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:42.358741 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71073f8e-94d8-4013-95a2-fa72af5f20ed None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:42.359037 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71073f8e-94d8-4013-95a2-fa72af5f20ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:42.380259 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c95e3a3-3460-4262-aaaa-3685a45d9f27 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:42.382134 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-71073f8e-94d8-4013-95a2-fa72af5f20ed None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:05:42.383643 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-71073f8e-94d8-4013-95a2-fa72af5f20ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:05:42.419957 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:42.434204 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eafdf3b-ffd1-4f49-8bbb-56e718d499ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:42.434899 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eafdf3b-ffd1-4f49-8bbb-56e718d499ad None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:42.435226 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eafdf3b-ffd1-4f49-8bbb-56e718d499ad None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:42.435619 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eafdf3b-ffd1-4f49-8bbb-56e718d499ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:42.436526 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eafdf3b-ffd1-4f49-8bbb-56e718d499ad None admin] RBAC: Adding query filter params (role.id=56205a5df6514298a69fde673550f0aa, scope.project.id=e192ac889bad428985088eea5ce750ab, user.id=639cd4e0d5ca4e4d8f6bab272d48a607) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:42.436942 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eafdf3b-ffd1-4f49-8bbb-56e718d499ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:42.438510 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eafdf3b-ffd1-4f49-8bbb-56e718d499ad None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:42.447309 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 55/120] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 4 09:05:42 2022] GET /identity/v3/role_assignments?user.id=639cd4e0d5ca4e4d8f6bab272d48a607&scope.project.id=e192ac889bad428985088eea5ce750ab&role.id=56205a5df6514298a69fde673550f0aa => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:42.452258 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7fabb4-2eab-4202-a8ee-a2e5aa423829 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:42.452258 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7fabb4-2eab-4202-a8ee-a2e5aa423829 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:42.452258 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7fabb4-2eab-4202-a8ee-a2e5aa423829 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:42.452258 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 56/121] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:05:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:05:42.452258 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7fabb4-2eab-4202-a8ee-a2e5aa423829 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:42.485457 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:42.493026 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 66/122] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:05:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:05:42.496311 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f646feba-0d76-45fa-b2c7-4f03b70a88a2 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:42.496807 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f646feba-0d76-45fa-b2c7-4f03b70a88a2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:42.497476 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f646feba-0d76-45fa-b2c7-4f03b70a88a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:42.520097 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f646feba-0d76-45fa-b2c7-4f03b70a88a2 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:05:42.521519 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f646feba-0d76-45fa-b2c7-4f03b70a88a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:05:42.589140 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7fabb4-2eab-4202-a8ee-a2e5aa423829 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:42.612600 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 67/123] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:05:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:05:42.613816 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f646feba-0d76-45fa-b2c7-4f03b70a88a2 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:42.677830 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c69220-bcc1-4b4a-9f6d-a385fa1be308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:42.678841 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c69220-bcc1-4b4a-9f6d-a385fa1be308 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:42.679284 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c69220-bcc1-4b4a-9f6d-a385fa1be308 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:42.679718 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c69220-bcc1-4b4a-9f6d-a385fa1be308 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:42.680694 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c69220-bcc1-4b4a-9f6d-a385fa1be308 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:42.681188 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c69220-bcc1-4b4a-9f6d-a385fa1be308 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:42.689082 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c69220-bcc1-4b4a-9f6d-a385fa1be308 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:42.696324 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 57/124] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:05:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:42.697077 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c69220-bcc1-4b4a-9f6d-a385fa1be308 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:42.737432 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f646feba-0d76-45fa-b2c7-4f03b70a88a2 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:42.770310 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:42.777395 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3027476-9e0a-492a-88f3-7b084b9e5481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:42.778465 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3027476-9e0a-492a-88f3-7b084b9e5481 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:42.778953 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3027476-9e0a-492a-88f3-7b084b9e5481 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:42.779493 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3027476-9e0a-492a-88f3-7b084b9e5481 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:42.781430 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:42.792877 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3027476-9e0a-492a-88f3-7b084b9e5481 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:05:42.802258 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-d3027476-9e0a-492a-88f3-7b084b9e5481 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:05:42.802258 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:42.802258 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:42.802258 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:42.802258 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:42.802258 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:42.802258 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:42.802258 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:42.802258 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:42.802258 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:42.802258 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:42.802258 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:42.802258 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:05:42.802258 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:05:42.802258 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:05:42.802258 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:05:42.803160 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:05:42.803160 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:05:42.803160 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:05:42.803160 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:05:42.803160 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:42.803160 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:42.803160 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:42.803160 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:42.803160 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:42.803160 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:42.803160 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:42.803160 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:42.803160 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:42.803160 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:42.803160 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:42.803160 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:42.803989 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:42.803989 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:42.803989 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:42.803989 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:42.803989 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:05:42.803989 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:05:42.803989 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:05:42.803989 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:05:42.803989 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:05:42.803989 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:05:42.803989 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:05:42.803989 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:05:42.805408 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 68/125] 10.222.0.71 () {56 vars in 1114 bytes} [Mon Jul 4 09:05:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:05:42.805408 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3027476-9e0a-492a-88f3-7b084b9e5481 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:42.898589 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c69220-bcc1-4b4a-9f6d-a385fa1be308 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:42.918796 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3027476-9e0a-492a-88f3-7b084b9e5481 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:42.955770 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1acd6ae-ecf2-4107-8330-0556795622ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:42.956653 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1acd6ae-ecf2-4107-8330-0556795622ad None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:42.956943 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1acd6ae-ecf2-4107-8330-0556795622ad None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:42.957257 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1acd6ae-ecf2-4107-8330-0556795622ad None admin] PATH_INFO: `/v3/users/fa43c0e65110476d9e52a93b45c1a988` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:42.958627 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1acd6ae-ecf2-4107-8330-0556795622ad None admin] RBAC: Authorizing `identity:get_user(user_id=fa43c0e65110476d9e52a93b45c1a988)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:42.960089 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1acd6ae-ecf2-4107-8330-0556795622ad None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:42.962604 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 69/126] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 4 09:05:42 2022] GET /identity/v3/users/fa43c0e65110476d9e52a93b45c1a988 => generated 247 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:42.963452 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1acd6ae-ecf2-4107-8330-0556795622ad None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:43.024938 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9d5298-e53f-46f2-b63e-5ec5f3574f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:43.026199 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9d5298-e53f-46f2-b63e-5ec5f3574f2a None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:43.026732 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9d5298-e53f-46f2-b63e-5ec5f3574f2a None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:43.027224 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9d5298-e53f-46f2-b63e-5ec5f3574f2a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:43.033518 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:43.069667 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9d5298-e53f-46f2-b63e-5ec5f3574f2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:05:43.073950 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1acd6ae-ecf2-4107-8330-0556795622ad None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:43.077888 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-3a9d5298-e53f-46f2-b63e-5ec5f3574f2a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:05:43.077888 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:43.077888 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:43.077888 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:43.077888 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:43.077888 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:43.077888 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:43.077888 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:43.077888 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:43.077888 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:43.077888 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:43.077888 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:43.077888 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:05:43.077888 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:05:43.077888 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:05:43.077888 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:05:43.078799 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:05:43.078799 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:05:43.078799 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:05:43.078799 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:05:43.078799 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:43.078799 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:43.078799 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:43.078799 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:43.078799 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:43.078799 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:43.078799 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:43.078799 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:43.078799 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:43.078799 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:43.078799 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:43.078799 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:43.079589 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:43.079589 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:43.079589 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:43.079589 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:43.079589 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:05:43.079589 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:05:43.079589 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:05:43.079589 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:05:43.079589 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:05:43.079589 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:05:43.079589 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:05:43.079589 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:05:43.079589 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 58/127] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:05:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 382 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:05:43.080247 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9d5298-e53f-46f2-b63e-5ec5f3574f2a None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:43.131270 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf3dd00-f4e1-49c7-968d-9a8d8943ab0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:43.133195 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf3dd00-f4e1-49c7-968d-9a8d8943ab0a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:43.133953 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf3dd00-f4e1-49c7-968d-9a8d8943ab0a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:43.134695 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf3dd00-f4e1-49c7-968d-9a8d8943ab0a None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:43.137996 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf3dd00-f4e1-49c7-968d-9a8d8943ab0a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:43.138546 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf3dd00-f4e1-49c7-968d-9a8d8943ab0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:43.143410 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf3dd00-f4e1-49c7-968d-9a8d8943ab0a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:43.160507 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 70/128] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 4 09:05:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:43.168658 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf3dd00-f4e1-49c7-968d-9a8d8943ab0a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:43.305181 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf3dd00-f4e1-49c7-968d-9a8d8943ab0a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:43.330300 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:43.331223 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9d5298-e53f-46f2-b63e-5ec5f3574f2a None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:43.358791 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb78e9b-2a85-4c7b-be49-38a704ae6869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:43.359838 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb78e9b-2a85-4c7b-be49-38a704ae6869 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:43.360349 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb78e9b-2a85-4c7b-be49-38a704ae6869 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:43.360844 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb78e9b-2a85-4c7b-be49-38a704ae6869 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:43.361409 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:43.361867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb78e9b-2a85-4c7b-be49-38a704ae6869 None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:43.362500 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb78e9b-2a85-4c7b-be49-38a704ae6869 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:43.363573 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb78e9b-2a85-4c7b-be49-38a704ae6869 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:43.402494 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 71/129] 10.222.0.71 () {56 vars in 1101 bytes} [Mon Jul 4 09:05:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:43.403678 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581aaf15-e8c0-44b8-89e4-ab8d36eb2239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:43.404574 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581aaf15-e8c0-44b8-89e4-ab8d36eb2239 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:43.404870 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581aaf15-e8c0-44b8-89e4-ab8d36eb2239 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:43.405187 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581aaf15-e8c0-44b8-89e4-ab8d36eb2239 None admin] PATH_INFO: `/v3/projects/e192ac889bad428985088eea5ce750ab` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:43.406127 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb78e9b-2a85-4c7b-be49-38a704ae6869 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:43.407152 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581aaf15-e8c0-44b8-89e4-ab8d36eb2239 None admin] RBAC: Authorizing `identity:get_project(project_id=e192ac889bad428985088eea5ce750ab)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:43.417367 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581aaf15-e8c0-44b8-89e4-ab8d36eb2239 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:43.419554 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 59/130] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 4 09:05:43 2022] GET /identity/v3/projects/e192ac889bad428985088eea5ce750ab => generated 298 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:43.420401 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581aaf15-e8c0-44b8-89e4-ab8d36eb2239 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:43.547351 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb78e9b-2a85-4c7b-be49-38a704ae6869 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:43.597760 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3c6aa9-fb20-42af-a6b3-3bbc3e80ba8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:43.598668 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c6aa9-fb20-42af-a6b3-3bbc3e80ba8b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:43.598965 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c6aa9-fb20-42af-a6b3-3bbc3e80ba8b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:43.599256 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c6aa9-fb20-42af-a6b3-3bbc3e80ba8b None admin] PATH_INFO: `/v3/users/a8b858d3091e4e51b82bacac35f8daa7` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:43.600505 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3c6aa9-fb20-42af-a6b3-3bbc3e80ba8b None admin] RBAC: Authorizing `identity:get_user(user_id=a8b858d3091e4e51b82bacac35f8daa7)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:43.602083 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3c6aa9-fb20-42af-a6b3-3bbc3e80ba8b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:43.603786 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 72/131] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 4 09:05:43 2022] GET /identity/v3/users/a8b858d3091e4e51b82bacac35f8daa7 => generated 275 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:43.604644 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3c6aa9-fb20-42af-a6b3-3bbc3e80ba8b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:43.632592 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581aaf15-e8c0-44b8-89e4-ab8d36eb2239 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:43.677651 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:43.718590 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b0588a-bf67-4ddb-aea8-d84dd0df074d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:43.719610 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b0588a-bf67-4ddb-aea8-d84dd0df074d None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:43.719909 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b0588a-bf67-4ddb-aea8-d84dd0df074d None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:43.720200 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b0588a-bf67-4ddb-aea8-d84dd0df074d None admin] PATH_INFO: `/v3/users/a8b858d3091e4e51b82bacac35f8daa7` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:43.724380 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b0588a-bf67-4ddb-aea8-d84dd0df074d None admin] RBAC: Authorizing `identity:get_user(user_id=a8b858d3091e4e51b82bacac35f8daa7)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:43.725897 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b0588a-bf67-4ddb-aea8-d84dd0df074d None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:43.727619 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 60/132] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 4 09:05:43 2022] GET /identity/v3/users/a8b858d3091e4e51b82bacac35f8daa7 => generated 275 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:43.728405 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b0588a-bf67-4ddb-aea8-d84dd0df074d None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:43.731897 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3c6aa9-fb20-42af-a6b3-3bbc3e80ba8b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:43.769702 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13c3b62-4df4-497f-8c12-dbe93796b30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:43.770753 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c3b62-4df4-497f-8c12-dbe93796b30f None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:43.771050 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c3b62-4df4-497f-8c12-dbe93796b30f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:43.771340 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c3b62-4df4-497f-8c12-dbe93796b30f None admin] PATH_INFO: `/v3/projects/e192ac889bad428985088eea5ce750ab/users/fa43c0e65110476d9e52a93b45c1a988/roles/f36e053b442540f08041ac7878760f65` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:43.773662 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13c3b62-4df4-497f-8c12-dbe93796b30f None admin] RBAC: Authorizing `identity:create_grant(project_id=e192ac889bad428985088eea5ce750ab, user_id=fa43c0e65110476d9e52a93b45c1a988, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:43.775568 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13c3b62-4df4-497f-8c12-dbe93796b30f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:43.804014 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 73/133] 10.222.0.71 () {58 vars in 1439 bytes} [Mon Jul 4 09:05:43 2022] PUT /identity/v3/projects/e192ac889bad428985088eea5ce750ab/users/fa43c0e65110476d9e52a93b45c1a988/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:05:43.804999 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13c3b62-4df4-497f-8c12-dbe93796b30f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:43.841295 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:43.955220 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b0588a-bf67-4ddb-aea8-d84dd0df074d None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:44.006761 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13c3b62-4df4-497f-8c12-dbe93796b30f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:44.037582 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:44.045818 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1418b8-c5bb-4437-8c07-f995583bcfe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:44.046915 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1418b8-c5bb-4437-8c07-f995583bcfe2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:44.047213 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1418b8-c5bb-4437-8c07-f995583bcfe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:44.047501 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1418b8-c5bb-4437-8c07-f995583bcfe2 None admin] PATH_INFO: `/v3/projects/de987c0b1cd242759308014bda6eb4bd` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:44.048930 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1418b8-c5bb-4437-8c07-f995583bcfe2 None admin] RBAC: Authorizing `identity:get_project(project_id=de987c0b1cd242759308014bda6eb4bd)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:44.050677 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1418b8-c5bb-4437-8c07-f995583bcfe2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:44.052966 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 74/134] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 4 09:05:43 2022] GET /identity/v3/projects/de987c0b1cd242759308014bda6eb4bd => generated 312 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:44.054756 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601dc373-71c9-4e9d-808f-2f6bf19bb538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:44.055706 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601dc373-71c9-4e9d-808f-2f6bf19bb538 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:44.055995 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601dc373-71c9-4e9d-808f-2f6bf19bb538 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:44.056280 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601dc373-71c9-4e9d-808f-2f6bf19bb538 None admin] PATH_INFO: `/v3/projects/e192ac889bad428985088eea5ce750ab` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:44.058782 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1418b8-c5bb-4437-8c07-f995583bcfe2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:44.064936 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601dc373-71c9-4e9d-808f-2f6bf19bb538 None admin] RBAC: Authorizing `identity:get_project(project_id=e192ac889bad428985088eea5ce750ab)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:44.067786 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601dc373-71c9-4e9d-808f-2f6bf19bb538 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:44.077539 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 61/135] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 4 09:05:43 2022] GET /identity/v3/projects/e192ac889bad428985088eea5ce750ab => generated 298 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:44.090769 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601dc373-71c9-4e9d-808f-2f6bf19bb538 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:44.200096 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1418b8-c5bb-4437-8c07-f995583bcfe2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:44.239752 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d3614c-0d14-4327-a811-ed603001e560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:44.240658 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d3614c-0d14-4327-a811-ed603001e560 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:44.240943 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d3614c-0d14-4327-a811-ed603001e560 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:44.241405 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d3614c-0d14-4327-a811-ed603001e560 None admin] PATH_INFO: `/v3/projects/de987c0b1cd242759308014bda6eb4bd/users/a8b858d3091e4e51b82bacac35f8daa7/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:44.243439 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d3614c-0d14-4327-a811-ed603001e560 None admin] RBAC: Authorizing `identity:create_grant(project_id=de987c0b1cd242759308014bda6eb4bd, user_id=a8b858d3091e4e51b82bacac35f8daa7, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:44.245256 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d3614c-0d14-4327-a811-ed603001e560 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:44.267879 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 75/136] 10.222.0.71 () {58 vars in 1439 bytes} [Mon Jul 4 09:05:44 2022] PUT /identity/v3/projects/de987c0b1cd242759308014bda6eb4bd/users/a8b858d3091e4e51b82bacac35f8daa7/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:05:44.277913 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601dc373-71c9-4e9d-808f-2f6bf19bb538 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:44.301480 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:44.321502 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca8a5fd-9aee-4bf0-b58c-19dfa6b64689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:44.322723 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8a5fd-9aee-4bf0-b58c-19dfa6b64689 None admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:44.323132 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8a5fd-9aee-4bf0-b58c-19dfa6b64689 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:44.323513 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8a5fd-9aee-4bf0-b58c-19dfa6b64689 None admin] PATH_INFO: `/v3/projects/e192ac889bad428985088eea5ce750ab/users/a8b858d3091e4e51b82bacac35f8daa7/roles/5fb58444d5604bf689014e27dccacb45` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:44.326225 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca8a5fd-9aee-4bf0-b58c-19dfa6b64689 None admin] RBAC: Authorizing `identity:create_grant(project_id=e192ac889bad428985088eea5ce750ab, user_id=a8b858d3091e4e51b82bacac35f8daa7, role_id=5fb58444d5604bf689014e27dccacb45)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:44.327822 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca8a5fd-9aee-4bf0-b58c-19dfa6b64689 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:44.356008 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 62/137] 10.222.0.71 () {58 vars in 1439 bytes} [Mon Jul 4 09:05:44 2022] PUT /identity/v3/projects/e192ac889bad428985088eea5ce750ab/users/a8b858d3091e4e51b82bacac35f8daa7/roles/5fb58444d5604bf689014e27dccacb45 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:05:44.674202 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c44d0e-7783-400c-853a-13dcde1e20d2 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:44.674717 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c44d0e-7783-400c-853a-13dcde1e20d2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:44.674717 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c44d0e-7783-400c-853a-13dcde1e20d2 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:44.676061 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 76/138] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:05:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:05:44.681515 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519af315-4c01-4a98-b1b8-b6d4bc9865f4 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:44.682504 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519af315-4c01-4a98-b1b8-b6d4bc9865f4 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:44.682719 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519af315-4c01-4a98-b1b8-b6d4bc9865f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:44.700340 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-519af315-4c01-4a98-b1b8-b6d4bc9865f4 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:05:44.701647 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-519af315-4c01-4a98-b1b8-b6d4bc9865f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:05:44.792847 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 63/139] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:05:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:05:44.831778 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c44d0e-7783-400c-853a-13dcde1e20d2 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:44.882440 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:44.918548 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf75535-e44c-4f43-8d4f-708abe4e169f None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:44.918872 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf75535-e44c-4f43-8d4f-708abe4e169f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:44.919125 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf75535-e44c-4f43-8d4f-708abe4e169f None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:44.920520 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 64/140] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:05:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:05:44.939069 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b644b5-3bea-4d25-97e4-b8d79f7525f8 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:44.939471 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b644b5-3bea-4d25-97e4-b8d79f7525f8 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:44.939945 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b644b5-3bea-4d25-97e4-b8d79f7525f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:44.971558 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-49b644b5-3bea-4d25-97e4-b8d79f7525f8 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:05:44.972781 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-49b644b5-3bea-4d25-97e4-b8d79f7525f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:05:44.998154 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c44d0e-7783-400c-853a-13dcde1e20d2 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:45.082392 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fcf241-6187-4c3e-9b38-cfdea443f14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:45.084448 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fcf241-6187-4c3e-9b38-cfdea443f14c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:45.084752 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fcf241-6187-4c3e-9b38-cfdea443f14c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:45.085032 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fcf241-6187-4c3e-9b38-cfdea443f14c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:45.087730 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fcf241-6187-4c3e-9b38-cfdea443f14c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:45.089437 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fcf241-6187-4c3e-9b38-cfdea443f14c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:45.091094 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 77/141] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:05:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:45.096278 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5867976f-1d4b-4f04-acb6-2632c871dd48 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:45.096570 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5867976f-1d4b-4f04-acb6-2632c871dd48 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:45.096851 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5867976f-1d4b-4f04-acb6-2632c871dd48 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:45.098311 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 78/142] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:05:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:05:45.101429 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5867976f-1d4b-4f04-acb6-2632c871dd48 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:45.122324 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 65/143] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:05:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:05:45.130346 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e6cd8-03d0-4ec9-8f5c-ab52a2de05fd None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:45.130647 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e6cd8-03d0-4ec9-8f5c-ab52a2de05fd None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:45.130940 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e6cd8-03d0-4ec9-8f5c-ab52a2de05fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:45.163751 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-c79e6cd8-03d0-4ec9-8f5c-ab52a2de05fd None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:05:45.164950 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c79e6cd8-03d0-4ec9-8f5c-ab52a2de05fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:05:45.229746 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:45.296209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 66/144] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:05:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:05:45.297090 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e6cd8-03d0-4ec9-8f5c-ab52a2de05fd None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:45.325933 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5867976f-1d4b-4f04-acb6-2632c871dd48 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:45.342261 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:45.369899 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad50d34-a979-4e13-96ed-4127a0b3e5eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:45.370987 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad50d34-a979-4e13-96ed-4127a0b3e5eb None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:45.371471 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad50d34-a979-4e13-96ed-4127a0b3e5eb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:45.371970 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad50d34-a979-4e13-96ed-4127a0b3e5eb None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:45.372878 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad50d34-a979-4e13-96ed-4127a0b3e5eb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:45.374656 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad50d34-a979-4e13-96ed-4127a0b3e5eb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:45.415925 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 79/145] 10.222.0.71 () {60 vars in 1132 bytes} [Mon Jul 4 09:05:45 2022] POST /identity/v3/users => generated 283 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:05:45.417887 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad50d34-a979-4e13-96ed-4127a0b3e5eb None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:45.502295 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e6cd8-03d0-4ec9-8f5c-ab52a2de05fd None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:45.595274 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f5b674-c0d3-4b3e-8262-2a7892a650c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:45.596129 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f5b674-c0d3-4b3e-8262-2a7892a650c6 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:45.596407 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f5b674-c0d3-4b3e-8262-2a7892a650c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:45.596691 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f5b674-c0d3-4b3e-8262-2a7892a650c6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:45.609801 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad50d34-a979-4e13-96ed-4127a0b3e5eb None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:45.629875 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f5b674-c0d3-4b3e-8262-2a7892a650c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:05:45.646571 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-94f5b674-c0d3-4b3e-8262-2a7892a650c6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:05:45.646571 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:45.646571 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:45.646571 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:45.646571 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:45.646571 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:45.646571 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:45.646571 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:45.646571 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:45.646571 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:45.646571 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:45.646571 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:45.646571 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:05:45.646571 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:05:45.646571 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:05:45.646571 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:05:45.647562 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:05:45.647562 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:05:45.647562 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:05:45.647562 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:05:45.647562 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:45.647562 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:45.647562 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:45.647562 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:45.647562 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:45.647562 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:45.647562 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:45.647562 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:45.647562 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:45.647562 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:45.647562 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:45.647562 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:45.648339 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:45.648339 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:45.648339 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:45.648339 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:45.648339 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:05:45.648339 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:05:45.648339 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:05:45.648339 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:05:45.648339 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:05:45.648339 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:05:45.648339 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:05:45.648339 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:05:45.648339 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 67/146] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:05:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:05:45.655363 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f5b674-c0d3-4b3e-8262-2a7892a650c6 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:45.681448 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b500d22-d127-462c-9083-b4bc45c519c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:45.683751 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b500d22-d127-462c-9083-b4bc45c519c3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:45.686005 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b500d22-d127-462c-9083-b4bc45c519c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:45.686625 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b500d22-d127-462c-9083-b4bc45c519c3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:45.713996 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b500d22-d127-462c-9083-b4bc45c519c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:05:45.724652 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:45.732297 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-2b500d22-d127-462c-9083-b4bc45c519c3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 09:05:45.732297 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:45.732297 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:45.732297 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:45.732297 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:45.732297 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:45.732297 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:45.732297 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:45.732297 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:45.732297 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:45.732297 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:45.732297 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:45.732297 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:05:45.732297 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:05:45.732297 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:05:45.732297 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:05:45.733211 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:05:45.733211 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:05:45.733211 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:05:45.733211 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:05:45.733211 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:45.733211 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:45.733211 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:45.733211 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:45.733211 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:45.733211 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:45.733211 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:45.733211 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:45.733211 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:45.733211 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:45.733211 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:45.733211 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:45.733994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:45.733994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:45.733994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:45.733994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:45.733994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:05:45.733994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:05:45.733994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:05:45.733994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:05:45.733994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:05:45.733994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:05:45.733994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 09:05:45.733994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:05:45.736484 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 80/147] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:05:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:05:45.742150 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b500d22-d127-462c-9083-b4bc45c519c3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:45.776190 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:45.876975 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b500d22-d127-462c-9083-b4bc45c519c3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:45.921842 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc9d730-98b6-40b9-a291-ebf446c95d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:45.922881 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc9d730-98b6-40b9-a291-ebf446c95d36 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:45.922987 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f5b674-c0d3-4b3e-8262-2a7892a650c6 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:45.923480 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc9d730-98b6-40b9-a291-ebf446c95d36 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:45.923966 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc9d730-98b6-40b9-a291-ebf446c95d36 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:45.926074 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc9d730-98b6-40b9-a291-ebf446c95d36 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:45.926599 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc9d730-98b6-40b9-a291-ebf446c95d36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:45.928195 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc9d730-98b6-40b9-a291-ebf446c95d36 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:45.943513 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 81/148] 10.222.0.71 () {56 vars in 1101 bytes} [Mon Jul 4 09:05:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:05:45.951293 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc9d730-98b6-40b9-a291-ebf446c95d36 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:45.993762 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d26c2b-6cdb-49ca-9b7d-54fbea50460f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:45.995755 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d26c2b-6cdb-49ca-9b7d-54fbea50460f None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:45.997283 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d26c2b-6cdb-49ca-9b7d-54fbea50460f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:45.997859 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d26c2b-6cdb-49ca-9b7d-54fbea50460f None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:46.000037 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d26c2b-6cdb-49ca-9b7d-54fbea50460f None admin] RBAC: Adding query filter params (name=member) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:46.000613 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d26c2b-6cdb-49ca-9b7d-54fbea50460f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:46.004005 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d26c2b-6cdb-49ca-9b7d-54fbea50460f None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:46.020743 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 68/149] 10.222.0.71 () {56 vars in 1101 bytes} [Mon Jul 4 09:05:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:05:46.030103 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d26c2b-6cdb-49ca-9b7d-54fbea50460f None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:46.125221 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:46.154937 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc9d730-98b6-40b9-a291-ebf446c95d36 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:46.209572 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:46.227811 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c151831-a4bd-4658-92cd-f2b1004d2c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:46.229376 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c151831-a4bd-4658-92cd-f2b1004d2c5c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:46.229513 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c151831-a4bd-4658-92cd-f2b1004d2c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:46.229793 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c151831-a4bd-4658-92cd-f2b1004d2c5c None admin] PATH_INFO: `/v3/users/639cd4e0d5ca4e4d8f6bab272d48a607` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:46.231701 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c151831-a4bd-4658-92cd-f2b1004d2c5c None admin] RBAC: Authorizing `identity:get_user(user_id=639cd4e0d5ca4e4d8f6bab272d48a607)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:46.233207 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c151831-a4bd-4658-92cd-f2b1004d2c5c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:46.233937 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d26c2b-6cdb-49ca-9b7d-54fbea50460f None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:46.234935 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 82/150] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 4 09:05:45 2022] GET /identity/v3/users/639cd4e0d5ca4e4d8f6bab272d48a607 => generated 289 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:46.241385 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c151831-a4bd-4658-92cd-f2b1004d2c5c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:46.325864 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867ae968-0eac-4156-9243-4d2b238546a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:46.327113 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867ae968-0eac-4156-9243-4d2b238546a2 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:46.327608 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867ae968-0eac-4156-9243-4d2b238546a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:46.328110 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867ae968-0eac-4156-9243-4d2b238546a2 None admin] PATH_INFO: `/v3/users/a8b858d3091e4e51b82bacac35f8daa7` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:46.329611 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867ae968-0eac-4156-9243-4d2b238546a2 None admin] RBAC: Authorizing `identity:get_user(user_id=a8b858d3091e4e51b82bacac35f8daa7)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:46.331452 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867ae968-0eac-4156-9243-4d2b238546a2 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:46.333363 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 69/151] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 4 09:05:46 2022] GET /identity/v3/users/a8b858d3091e4e51b82bacac35f8daa7 => generated 275 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:46.339922 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867ae968-0eac-4156-9243-4d2b238546a2 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:46.371509 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c151831-a4bd-4658-92cd-f2b1004d2c5c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:46.423344 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feb38de-b881-4ce9-9ba0-f3a363771055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:46.424217 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb38de-b881-4ce9-9ba0-f3a363771055 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:46.424498 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb38de-b881-4ce9-9ba0-f3a363771055 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:46.424771 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb38de-b881-4ce9-9ba0-f3a363771055 None admin] PATH_INFO: `/v3/projects/e192ac889bad428985088eea5ce750ab` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:46.426007 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feb38de-b881-4ce9-9ba0-f3a363771055 None admin] RBAC: Authorizing `identity:get_project(project_id=e192ac889bad428985088eea5ce750ab)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:46.430494 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feb38de-b881-4ce9-9ba0-f3a363771055 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:46.430494 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 83/152] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 4 09:05:46 2022] GET /identity/v3/projects/e192ac889bad428985088eea5ce750ab => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:46.439324 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feb38de-b881-4ce9-9ba0-f3a363771055 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:46.470961 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:46.564257 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feb38de-b881-4ce9-9ba0-f3a363771055 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:46.565243 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867ae968-0eac-4156-9243-4d2b238546a2 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:46.606474 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28f9ea2-208a-4f98-a9aa-774d610851cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:46.607308 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f9ea2-208a-4f98-a9aa-774d610851cc None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:46.607650 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f9ea2-208a-4f98-a9aa-774d610851cc None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:46.607932 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f9ea2-208a-4f98-a9aa-774d610851cc None admin] PATH_INFO: `/v3/projects/e192ac889bad428985088eea5ce750ab/users/639cd4e0d5ca4e4d8f6bab272d48a607/roles/56205a5df6514298a69fde673550f0aa` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:46.610331 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28f9ea2-208a-4f98-a9aa-774d610851cc None admin] RBAC: Authorizing `identity:create_grant(project_id=e192ac889bad428985088eea5ce750ab, user_id=639cd4e0d5ca4e4d8f6bab272d48a607, role_id=56205a5df6514298a69fde673550f0aa)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:46.611954 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28f9ea2-208a-4f98-a9aa-774d610851cc None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:46.626941 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c8899f-20e0-4019-8bac-a85c553c5262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:46.627685 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c8899f-20e0-4019-8bac-a85c553c5262 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:46.627982 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c8899f-20e0-4019-8bac-a85c553c5262 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:46.628270 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c8899f-20e0-4019-8bac-a85c553c5262 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:46.632359 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 84/153] 10.222.0.71 () {58 vars in 1439 bytes} [Mon Jul 4 09:05:46 2022] PUT /identity/v3/projects/e192ac889bad428985088eea5ce750ab/users/639cd4e0d5ca4e4d8f6bab272d48a607/roles/56205a5df6514298a69fde673550f0aa => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:05:46.665690 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:46.731628 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c8899f-20e0-4019-8bac-a85c553c5262 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:46.734143 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c8899f-20e0-4019-8bac-a85c553c5262 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:46.856887 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 70/154] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:05:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 04 09:05:46.862639 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28f9ea2-208a-4f98-a9aa-774d610851cc None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:46.970304 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:46.992460 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28f9ea2-208a-4f98-a9aa-774d610851cc None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:47.028881 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf63742-c475-4660-aa4b-62ef995e287b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:47.029786 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf63742-c475-4660-aa4b-62ef995e287b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:47.030074 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf63742-c475-4660-aa4b-62ef995e287b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:47.030352 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf63742-c475-4660-aa4b-62ef995e287b None admin] PATH_INFO: `/v3/projects/e192ac889bad428985088eea5ce750ab` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:47.031562 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf63742-c475-4660-aa4b-62ef995e287b None admin] RBAC: Authorizing `identity:get_project(project_id=e192ac889bad428985088eea5ce750ab)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:47.033002 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf63742-c475-4660-aa4b-62ef995e287b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:47.034968 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 85/155] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 4 09:05:46 2022] GET /identity/v3/projects/e192ac889bad428985088eea5ce750ab => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:47.040873 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c8899f-20e0-4019-8bac-a85c553c5262 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:47.104699 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:47.151778 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c8899f-20e0-4019-8bac-a85c553c5262 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:47.162509 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e82065-2c5f-4c3b-933a-b1bf89ad81f0 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:47.162807 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e82065-2c5f-4c3b-933a-b1bf89ad81f0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:47.163092 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e82065-2c5f-4c3b-933a-b1bf89ad81f0 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:47.164611 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 86/156] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:05:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:05:47.177370 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a974d0fd-fc16-47c3-9063-cd4d93343617 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:47.177677 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a974d0fd-fc16-47c3-9063-cd4d93343617 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:47.177937 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a974d0fd-fc16-47c3-9063-cd4d93343617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:47.210158 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a974d0fd-fc16-47c3-9063-cd4d93343617 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:05:47.211392 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a974d0fd-fc16-47c3-9063-cd4d93343617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:05:47.238692 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0d3894-3b58-4e2f-8d24-af3f8f5bfb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:47.239369 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d3894-3b58-4e2f-8d24-af3f8f5bfb25 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:47.239667 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d3894-3b58-4e2f-8d24-af3f8f5bfb25 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:47.239946 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d3894-3b58-4e2f-8d24-af3f8f5bfb25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:47.240788 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0d3894-3b58-4e2f-8d24-af3f8f5bfb25 None admin] RBAC: Adding query filter params (role.id=8692e99e9463459f9b7caa96b3051b93, scope.project.id=e192ac889bad428985088eea5ce750ab, user.id=a8b858d3091e4e51b82bacac35f8daa7) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:47.241126 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0d3894-3b58-4e2f-8d24-af3f8f5bfb25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:47.242613 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0d3894-3b58-4e2f-8d24-af3f8f5bfb25 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:47.253955 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 71/157] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 4 09:05:47 2022] GET /identity/v3/role_assignments?user.id=a8b858d3091e4e51b82bacac35f8daa7&scope.project.id=e192ac889bad428985088eea5ce750ab&role.id=8692e99e9463459f9b7caa96b3051b93 => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:47.341561 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 87/158] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:05:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:05:47.357857 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92481530-8a24-4764-a008-51187773e403 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:47.358184 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92481530-8a24-4764-a008-51187773e403 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:47.358416 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92481530-8a24-4764-a008-51187773e403 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:47.359813 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 72/159] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:05:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:05:47.365739 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59788631-09ec-4f4a-b2f0-643eef58c9df None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:47.366039 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59788631-09ec-4f4a-b2f0-643eef58c9df None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:47.366323 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59788631-09ec-4f4a-b2f0-643eef58c9df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:47.374332 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92481530-8a24-4764-a008-51187773e403 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:47.387678 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-59788631-09ec-4f4a-b2f0-643eef58c9df None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:05:47.388893 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-59788631-09ec-4f4a-b2f0-643eef58c9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:05:47.507866 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 88/160] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:05:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:05:47.555630 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59788631-09ec-4f4a-b2f0-643eef58c9df None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:47.577229 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:47.620298 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92481530-8a24-4764-a008-51187773e403 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:47.662524 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:47.707158 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886aad16-20dc-4a00-93c6-7720c8a9042e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:47.707975 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886aad16-20dc-4a00-93c6-7720c8a9042e None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:47.708260 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886aad16-20dc-4a00-93c6-7720c8a9042e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:47.708540 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886aad16-20dc-4a00-93c6-7720c8a9042e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:47.733995 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886aad16-20dc-4a00-93c6-7720c8a9042e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:05:47.741745 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-886aad16-20dc-4a00-93c6-7720c8a9042e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:05:47.741745 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:47.741745 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:47.741745 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:47.741745 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:47.741745 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:47.741745 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:47.741745 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:47.741745 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:47.741745 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:47.741745 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:47.741745 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:47.741745 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:05:47.741745 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:05:47.741745 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:05:47.741745 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:05:47.742629 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:05:47.742629 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:05:47.742629 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:05:47.742629 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:05:47.742629 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:47.742629 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:47.742629 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:47.742629 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:47.742629 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:47.742629 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:47.742629 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:47.742629 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:47.742629 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:47.742629 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:47.742629 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:47.742629 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:47.743421 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:47.743421 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:47.743421 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:47.743421 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:47.743421 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:05:47.743421 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:05:47.743421 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:05:47.743421 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:05:47.743421 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:05:47.743421 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:05:47.743421 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:05:47.743421 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:05:47.743421 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 73/161] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 4 09:05:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:05:47.749308 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886aad16-20dc-4a00-93c6-7720c8a9042e None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:47.758291 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59788631-09ec-4f4a-b2f0-643eef58c9df None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:47.799050 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eaae0b-7a18-4bca-8c99-9912dd9f5556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:47.799976 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eaae0b-7a18-4bca-8c99-9912dd9f5556 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:47.800313 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eaae0b-7a18-4bca-8c99-9912dd9f5556 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:47.800597 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eaae0b-7a18-4bca-8c99-9912dd9f5556 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:47.812611 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eaae0b-7a18-4bca-8c99-9912dd9f5556 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:05:47.823821 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-86eaae0b-7a18-4bca-8c99-9912dd9f5556 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:05:47.823821 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:47.823821 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:47.823821 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:47.823821 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:47.823821 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:47.823821 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:47.823821 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:47.823821 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:47.823821 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:47.823821 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:47.823821 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:47.823821 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:05:47.823821 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:05:47.823821 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:05:47.823821 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:05:47.824746 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:05:47.824746 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:05:47.824746 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:05:47.824746 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:05:47.824746 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:47.824746 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:47.824746 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:47.824746 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:47.824746 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:47.824746 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:47.824746 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:47.824746 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:47.824746 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:47.824746 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:47.824746 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:47.824746 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:47.825621 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:47.825621 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:47.825621 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:47.825621 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:47.825621 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:05:47.825621 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:05:47.825621 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:05:47.825621 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:05:47.825621 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:05:47.825621 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:05:47.825621 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:05:47.825621 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:05:47.825621 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 89/162] 10.222.0.71 () {56 vars in 1114 bytes} [Mon Jul 4 09:05:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:05:47.831198 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eaae0b-7a18-4bca-8c99-9912dd9f5556 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:47.914032 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:47.964423 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eaae0b-7a18-4bca-8c99-9912dd9f5556 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:48.020963 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886aad16-20dc-4a00-93c6-7720c8a9042e None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:48.021603 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2a4baa-6b35-4231-a23e-ad2ea9060eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:48.022459 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a4baa-6b35-4231-a23e-ad2ea9060eb3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:48.022752 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a4baa-6b35-4231-a23e-ad2ea9060eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:48.023080 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a4baa-6b35-4231-a23e-ad2ea9060eb3 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:48.023847 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2a4baa-6b35-4231-a23e-ad2ea9060eb3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:48.024160 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2a4baa-6b35-4231-a23e-ad2ea9060eb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:48.026124 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2a4baa-6b35-4231-a23e-ad2ea9060eb3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:48.043053 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 90/163] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 4 09:05:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:48.045807 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2a4baa-6b35-4231-a23e-ad2ea9060eb3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:48.092493 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9133ac84-3501-432e-84cf-1c0bf04bd205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:48.093201 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133ac84-3501-432e-84cf-1c0bf04bd205 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:48.093725 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133ac84-3501-432e-84cf-1c0bf04bd205 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:48.094011 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133ac84-3501-432e-84cf-1c0bf04bd205 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:48.094773 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9133ac84-3501-432e-84cf-1c0bf04bd205 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:48.095970 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9133ac84-3501-432e-84cf-1c0bf04bd205 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:48.098110 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9133ac84-3501-432e-84cf-1c0bf04bd205 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:48.120193 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 74/164] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:05:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:05:48.129586 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9133ac84-3501-432e-84cf-1c0bf04bd205 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:48.209822 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:48.231063 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2a4baa-6b35-4231-a23e-ad2ea9060eb3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:48.264879 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:48.274029 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af4f38e-a725-4510-a89d-82b9548ce2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:48.276177 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4f38e-a725-4510-a89d-82b9548ce2af None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:48.276617 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4f38e-a725-4510-a89d-82b9548ce2af None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:48.277084 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4f38e-a725-4510-a89d-82b9548ce2af None admin] PATH_INFO: `/v3/users/a8b858d3091e4e51b82bacac35f8daa7` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:48.278500 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af4f38e-a725-4510-a89d-82b9548ce2af None admin] RBAC: Authorizing `identity:get_user(user_id=a8b858d3091e4e51b82bacac35f8daa7)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:48.280174 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af4f38e-a725-4510-a89d-82b9548ce2af None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:48.282661 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 91/165] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 4 09:05:48 2022] GET /identity/v3/users/a8b858d3091e4e51b82bacac35f8daa7 => generated 275 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:05:48.286933 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac5979c-16c0-44c3-9de0-9989a739a07c None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:48.287403 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac5979c-16c0-44c3-9de0-9989a739a07c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:48.287828 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac5979c-16c0-44c3-9de0-9989a739a07c None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:48.290043 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 92/166] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:05:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:05:48.294919 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac5979c-16c0-44c3-9de0-9989a739a07c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:48.347587 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9133ac84-3501-432e-84cf-1c0bf04bd205 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:48.423476 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2638ced-917c-4f42-94cf-eb82d4a5c8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:48.424375 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2638ced-917c-4f42-94cf-eb82d4a5c8e9 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:48.424657 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2638ced-917c-4f42-94cf-eb82d4a5c8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:48.424934 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2638ced-917c-4f42-94cf-eb82d4a5c8e9 None admin] PATH_INFO: `/v3/users/fa43c0e65110476d9e52a93b45c1a988` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:48.426259 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2638ced-917c-4f42-94cf-eb82d4a5c8e9 None admin] RBAC: Authorizing `identity:get_user(user_id=fa43c0e65110476d9e52a93b45c1a988)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:48.427721 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2638ced-917c-4f42-94cf-eb82d4a5c8e9 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:48.433164 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 75/167] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 4 09:05:48 2022] GET /identity/v3/users/fa43c0e65110476d9e52a93b45c1a988 => generated 247 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:48.434418 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de958990-30ba-405f-9799-5fcd7bdc0a0e None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:48.434707 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de958990-30ba-405f-9799-5fcd7bdc0a0e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:48.434992 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de958990-30ba-405f-9799-5fcd7bdc0a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:48.469521 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-de958990-30ba-405f-9799-5fcd7bdc0a0e None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:05:48.470791 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-de958990-30ba-405f-9799-5fcd7bdc0a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:05:48.490642 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac5979c-16c0-44c3-9de0-9989a739a07c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:48.548949 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 76/168] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:05:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:05:48.551306 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de958990-30ba-405f-9799-5fcd7bdc0a0e None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:48.559712 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a639890-6d0b-4a8a-be94-4821182eaf69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:48.562305 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a639890-6d0b-4a8a-be94-4821182eaf69 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:48.562592 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a639890-6d0b-4a8a-be94-4821182eaf69 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:48.562874 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a639890-6d0b-4a8a-be94-4821182eaf69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:48.621214 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:48.684070 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de958990-30ba-405f-9799-5fcd7bdc0a0e None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:48.724108 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2185a7-26dc-4a43-af22-ef0c85816ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:48.724793 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2185a7-26dc-4a43-af22-ef0c85816ca8 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:48.725181 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2185a7-26dc-4a43-af22-ef0c85816ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:48.725674 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2185a7-26dc-4a43-af22-ef0c85816ca8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:48.740343 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:48.749848 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a639890-6d0b-4a8a-be94-4821182eaf69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:48.751320 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a639890-6d0b-4a8a-be94-4821182eaf69 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:48.803880 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2185a7-26dc-4a43-af22-ef0c85816ca8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:48.805945 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2185a7-26dc-4a43-af22-ef0c85816ca8 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:48.875699 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:48.920547 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 77/169] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:05:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:05:48.922024 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2185a7-26dc-4a43-af22-ef0c85816ca8 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:48.932934 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 93/170] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:05:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 637 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 04 09:05:48.933455 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a639890-6d0b-4a8a-be94-4821182eaf69 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:49.040724 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:49.046852 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a639890-6d0b-4a8a-be94-4821182eaf69 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:49.080088 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2185a7-26dc-4a43-af22-ef0c85816ca8 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:49.093204 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e88266-f7cf-4969-b271-876622bdcaec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:49.094411 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e88266-f7cf-4969-b271-876622bdcaec None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:49.094901 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e88266-f7cf-4969-b271-876622bdcaec None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:49.095375 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e88266-f7cf-4969-b271-876622bdcaec None admin] PATH_INFO: `/v3/projects/e192ac889bad428985088eea5ce750ab` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:49.096821 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e88266-f7cf-4969-b271-876622bdcaec None admin] RBAC: Authorizing `identity:get_project(project_id=e192ac889bad428985088eea5ce750ab)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:49.098558 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e88266-f7cf-4969-b271-876622bdcaec None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:49.106348 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 94/171] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 4 09:05:48 2022] GET /identity/v3/projects/e192ac889bad428985088eea5ce750ab => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:49.107492 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e88266-f7cf-4969-b271-876622bdcaec None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:49.123924 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edba3e0-a15d-4f04-9929-412e49e0313a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:49.124746 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edba3e0-a15d-4f04-9929-412e49e0313a None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:49.125054 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edba3e0-a15d-4f04-9929-412e49e0313a None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:49.125368 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edba3e0-a15d-4f04-9929-412e49e0313a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:49.133826 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:49.141746 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edba3e0-a15d-4f04-9929-412e49e0313a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:05:49.152419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-9edba3e0-a15d-4f04-9929-412e49e0313a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:05:49.152419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:49.152419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:49.152419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:49.152419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:49.152419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:49.152419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:49.152419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:49.152419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:49.152419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:49.152419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:49.152419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:49.152419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:05:49.152419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:05:49.152419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:05:49.152419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:05:49.153372 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:05:49.153372 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:05:49.153372 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:05:49.153372 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:05:49.153372 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:49.153372 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:49.153372 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:49.153372 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:49.153372 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:49.153372 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:49.153372 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:49.153372 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:49.153372 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:49.153372 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:49.153372 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:49.153372 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:49.154265 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:49.154265 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:49.154265 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:49.154265 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:49.154265 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:05:49.154265 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:05:49.154265 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:05:49.154265 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:05:49.154265 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:05:49.154265 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:05:49.154265 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:05:49.154265 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:05:49.154265 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 78/172] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:05:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:05:49.154938 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edba3e0-a15d-4f04-9929-412e49e0313a None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:49.265833 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edba3e0-a15d-4f04-9929-412e49e0313a None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:49.302225 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0015e7e2-c838-46a3-8835-263169728ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:49.302901 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0015e7e2-c838-46a3-8835-263169728ad3 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:49.303195 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0015e7e2-c838-46a3-8835-263169728ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:49.303480 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0015e7e2-c838-46a3-8835-263169728ad3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:49.304326 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0015e7e2-c838-46a3-8835-263169728ad3 None admin] RBAC: Adding query filter params (role.id=f36e053b442540f08041ac7878760f65, scope.project.id=e192ac889bad428985088eea5ce750ab, user.id=fa43c0e65110476d9e52a93b45c1a988) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:49.304646 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0015e7e2-c838-46a3-8835-263169728ad3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:49.306287 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0015e7e2-c838-46a3-8835-263169728ad3 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:49.312980 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 79/173] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 4 09:05:49 2022] GET /identity/v3/role_assignments?user.id=fa43c0e65110476d9e52a93b45c1a988&scope.project.id=e192ac889bad428985088eea5ce750ab&role.id=f36e053b442540f08041ac7878760f65 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:49.318093 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0015e7e2-c838-46a3-8835-263169728ad3 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:49.345080 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e88266-f7cf-4969-b271-876622bdcaec None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:49.426574 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68696543-c1a9-42a3-8da7-c53e3ef9eecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:49.427480 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68696543-c1a9-42a3-8da7-c53e3ef9eecd None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:49.427762 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68696543-c1a9-42a3-8da7-c53e3ef9eecd None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:49.428039 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68696543-c1a9-42a3-8da7-c53e3ef9eecd None admin] PATH_INFO: `/v3/projects/e192ac889bad428985088eea5ce750ab` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:49.429288 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68696543-c1a9-42a3-8da7-c53e3ef9eecd None admin] RBAC: Authorizing `identity:get_project(project_id=e192ac889bad428985088eea5ce750ab)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:49.430738 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68696543-c1a9-42a3-8da7-c53e3ef9eecd None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:49.432904 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 95/174] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 4 09:05:49 2022] GET /identity/v3/projects/e192ac889bad428985088eea5ce750ab => generated 298 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:49.438695 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68696543-c1a9-42a3-8da7-c53e3ef9eecd None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:49.449449 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:49.489980 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0015e7e2-c838-46a3-8835-263169728ad3 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:49.521464 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:49.576948 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79edac1-7543-4780-a9c3-e1e546f6059e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:49.577970 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79edac1-7543-4780-a9c3-e1e546f6059e None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:49.578463 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79edac1-7543-4780-a9c3-e1e546f6059e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:49.578932 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79edac1-7543-4780-a9c3-e1e546f6059e None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:49.580002 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79edac1-7543-4780-a9c3-e1e546f6059e None admin] RBAC: Adding query filter params (name=member) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:49.580609 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79edac1-7543-4780-a9c3-e1e546f6059e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:49.582211 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79edac1-7543-4780-a9c3-e1e546f6059e None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:49.605967 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 80/175] 10.222.0.71 () {56 vars in 1101 bytes} [Mon Jul 4 09:05:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:49.614169 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79edac1-7543-4780-a9c3-e1e546f6059e None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:49.664408 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68696543-c1a9-42a3-8da7-c53e3ef9eecd None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:49.745112 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e9c41-8c44-4ec0-8da3-2053b60c2bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:49.745795 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e9c41-8c44-4ec0-8da3-2053b60c2bfe None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:49.746081 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e9c41-8c44-4ec0-8da3-2053b60c2bfe None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:49.746367 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e9c41-8c44-4ec0-8da3-2053b60c2bfe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:49.747512 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34e9c41-8c44-4ec0-8da3-2053b60c2bfe None admin] RBAC: Adding query filter params (role.id=5fb58444d5604bf689014e27dccacb45, scope.project.id=e192ac889bad428985088eea5ce750ab, user.id=a8b858d3091e4e51b82bacac35f8daa7) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:49.747869 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34e9c41-8c44-4ec0-8da3-2053b60c2bfe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:49.750539 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34e9c41-8c44-4ec0-8da3-2053b60c2bfe None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:49.769056 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 96/176] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 4 09:05:49 2022] GET /identity/v3/role_assignments?user.id=a8b858d3091e4e51b82bacac35f8daa7&scope.project.id=e192ac889bad428985088eea5ce750ab&role.id=5fb58444d5604bf689014e27dccacb45 => generated 608 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:05:49.785984 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79edac1-7543-4780-a9c3-e1e546f6059e None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:49.816079 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:49.827098 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660f26a0-1ff4-486a-a7ac-745e3f6cf60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:49.828098 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f26a0-1ff4-486a-a7ac-745e3f6cf60f None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:49.828364 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f26a0-1ff4-486a-a7ac-745e3f6cf60f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:49.828646 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f26a0-1ff4-486a-a7ac-745e3f6cf60f None admin] PATH_INFO: `/v3/users/a8b858d3091e4e51b82bacac35f8daa7` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:49.829956 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660f26a0-1ff4-486a-a7ac-745e3f6cf60f None admin] RBAC: Authorizing `identity:get_user(user_id=a8b858d3091e4e51b82bacac35f8daa7)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:49.831432 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660f26a0-1ff4-486a-a7ac-745e3f6cf60f None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:49.833159 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 81/177] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 4 09:05:49 2022] GET /identity/v3/users/a8b858d3091e4e51b82bacac35f8daa7 => generated 275 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:49.840077 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e9c41-8c44-4ec0-8da3-2053b60c2bfe None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:49.974512 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:49.984060 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e9c41-8c44-4ec0-8da3-2053b60c2bfe None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:50.032290 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c040fe-8b35-4adc-8c18-2001fda46e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:50.032972 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c040fe-8b35-4adc-8c18-2001fda46e83 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:50.033313 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c040fe-8b35-4adc-8c18-2001fda46e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:50.033599 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c040fe-8b35-4adc-8c18-2001fda46e83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:50.101729 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cafe91-1813-4371-931b-534cde958d11 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:50.102236 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cafe91-1813-4371-931b-534cde958d11 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:50.102669 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cafe91-1813-4371-931b-534cde958d11 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:50.104579 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 82/178] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:05:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:05:50.110540 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d8055-d020-407c-9643-78e669be8369 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:50.110836 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d8055-d020-407c-9643-78e669be8369 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:50.111130 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d8055-d020-407c-9643-78e669be8369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:50.146237 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c040fe-8b35-4adc-8c18-2001fda46e83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:50.148014 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c040fe-8b35-4adc-8c18-2001fda46e83 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:50.148574 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-099d8055-d020-407c-9643-78e669be8369 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:05:50.149842 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-099d8055-d020-407c-9643-78e669be8369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:05:50.271900 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 83/179] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:05:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:05:50.293420 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:50.297978 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 97/180] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:05:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:05:50.300573 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099d8055-d020-407c-9643-78e669be8369 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:50.315245 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c040fe-8b35-4adc-8c18-2001fda46e83 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:50.474479 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099d8055-d020-407c-9643-78e669be8369 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:50.494062 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c040fe-8b35-4adc-8c18-2001fda46e83 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:50.503377 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:50.515259 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7621e7-f920-4736-9f08-f120e0ce668a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:50.516075 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7621e7-f920-4736-9f08-f120e0ce668a None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:50.516361 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7621e7-f920-4736-9f08-f120e0ce668a None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:50.516645 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7621e7-f920-4736-9f08-f120e0ce668a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:50.527416 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7621e7-f920-4736-9f08-f120e0ce668a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:05:50.535280 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-7a7621e7-f920-4736-9f08-f120e0ce668a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:05:50.535280 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:50.535280 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:50.535280 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:50.535280 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:50.535280 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:50.535280 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:50.535280 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:50.535280 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:50.535280 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:50.535280 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:50.535280 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:50.535280 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:05:50.535280 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:05:50.535280 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:05:50.535280 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:05:50.536167 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:05:50.536167 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:05:50.536167 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:05:50.536167 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:05:50.536167 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:50.536167 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:50.536167 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:50.536167 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:50.536167 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:50.536167 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:50.536167 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:50.536167 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:50.536167 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:50.536167 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:50.536167 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:50.536167 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:50.536981 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:50.536981 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:50.536981 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:50.536981 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:50.536981 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:05:50.536981 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:05:50.536981 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:05:50.536981 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:05:50.536981 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:05:50.536981 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:05:50.536981 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:05:50.536981 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:05:50.536981 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 84/181] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 4 09:05:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:05:50.537687 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca3804-4cb0-4e87-a7e0-03ad9332599f None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:50.538495 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca3804-4cb0-4e87-a7e0-03ad9332599f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:50.538495 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca3804-4cb0-4e87-a7e0-03ad9332599f None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:50.544056 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24c8653-2202-4580-b9bf-c7f9b945bb16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:50.544749 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 85/182] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:05:50 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:05:50.545429 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c8653-2202-4580-b9bf-c7f9b945bb16 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:50.545901 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ca3804-4cb0-4e87-a7e0-03ad9332599f None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:50.546129 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c8653-2202-4580-b9bf-c7f9b945bb16 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:50.546601 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c8653-2202-4580-b9bf-c7f9b945bb16 None admin] PATH_INFO: `/v3/projects/de987c0b1cd242759308014bda6eb4bd` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:50.548133 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24c8653-2202-4580-b9bf-c7f9b945bb16 None admin] RBAC: Authorizing `identity:get_project(project_id=de987c0b1cd242759308014bda6eb4bd)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:50.549810 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24c8653-2202-4580-b9bf-c7f9b945bb16 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:50.551745 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 98/183] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 4 09:05:50 2022] GET /identity/v3/projects/de987c0b1cd242759308014bda6eb4bd => generated 312 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:50.559825 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6fa1b2-2380-472c-b3a2-10a156d2fc45 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:50.560428 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6fa1b2-2380-472c-b3a2-10a156d2fc45 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:50.560915 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6fa1b2-2380-472c-b3a2-10a156d2fc45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:50.581505 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3c6fa1b2-2380-472c-b3a2-10a156d2fc45 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:05:50.583207 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3c6fa1b2-2380-472c-b3a2-10a156d2fc45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:05:50.652737 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:50.661874 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 99/184] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:05:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:05:50.662797 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6fa1b2-2380-472c-b3a2-10a156d2fc45 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:50.678009 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ca3804-4cb0-4e87-a7e0-03ad9332599f None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:50.719123 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c86959-c291-4998-bac7-41f6898c21bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:50.719811 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c86959-c291-4998-bac7-41f6898c21bd None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:50.720245 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c86959-c291-4998-bac7-41f6898c21bd None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:50.720531 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c86959-c291-4998-bac7-41f6898c21bd None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:50.721319 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c86959-c291-4998-bac7-41f6898c21bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:50.721629 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c86959-c291-4998-bac7-41f6898c21bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:50.723036 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c86959-c291-4998-bac7-41f6898c21bd None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:50.732863 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 86/185] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:05:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:50.734395 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8289019e-e2f7-49f3-bf5b-7521c2d1e4e6 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:50.734838 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8289019e-e2f7-49f3-bf5b-7521c2d1e4e6 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:50.735300 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8289019e-e2f7-49f3-bf5b-7521c2d1e4e6 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:50.736695 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 87/186] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:05:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:05:50.737708 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8289019e-e2f7-49f3-bf5b-7521c2d1e4e6 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:50.776897 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6fa1b2-2380-472c-b3a2-10a156d2fc45 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:50.786176 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:50.812147 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8319df52-85d9-4862-a115-e9db2d087509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:50.812808 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8319df52-85d9-4862-a115-e9db2d087509 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:50.813127 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8319df52-85d9-4862-a115-e9db2d087509 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:50.813413 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8319df52-85d9-4862-a115-e9db2d087509 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:50.814260 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8319df52-85d9-4862-a115-e9db2d087509 None admin] RBAC: Adding query filter params (role.id=8692e99e9463459f9b7caa96b3051b93, scope.project.id=de987c0b1cd242759308014bda6eb4bd, user.id=a8b858d3091e4e51b82bacac35f8daa7) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:50.814578 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8319df52-85d9-4862-a115-e9db2d087509 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:50.816023 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8319df52-85d9-4862-a115-e9db2d087509 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:50.821753 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 100/187] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 4 09:05:50 2022] GET /identity/v3/role_assignments?user.id=a8b858d3091e4e51b82bacac35f8daa7&scope.project.id=de987c0b1cd242759308014bda6eb4bd&role.id=8692e99e9463459f9b7caa96b3051b93 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:50.828767 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1293ce75-2e15-4038-aa45-17d7a6876468 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:50.828767 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1293ce75-2e15-4038-aa45-17d7a6876468 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:50.828767 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1293ce75-2e15-4038-aa45-17d7a6876468 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:50.828767 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 101/188] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:05:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:05:50.828767 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1293ce75-2e15-4038-aa45-17d7a6876468 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:50.841434 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8289019e-e2f7-49f3-bf5b-7521c2d1e4e6 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:50.881316 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bb84e2-1b15-40cc-8047-ce4cbb7ad35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:50.882400 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb84e2-1b15-40cc-8047-ce4cbb7ad35e None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:50.882694 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb84e2-1b15-40cc-8047-ce4cbb7ad35e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:50.882980 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb84e2-1b15-40cc-8047-ce4cbb7ad35e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:50.895629 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bb84e2-1b15-40cc-8047-ce4cbb7ad35e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:05:50.901067 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:50.905248 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-21bb84e2-1b15-40cc-8047-ce4cbb7ad35e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:05:50.905248 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:50.905248 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:50.905248 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:50.905248 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:50.905248 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:50.905248 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:50.905248 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:50.905248 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:50.905248 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:50.905248 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:50.905248 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:50.905248 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:05:50.905248 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:05:50.905248 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:05:50.905248 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:05:50.906213 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:05:50.906213 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:05:50.906213 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:05:50.906213 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:05:50.906213 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:50.906213 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:50.906213 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:50.906213 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:50.906213 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:50.906213 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:50.906213 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:50.906213 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:50.906213 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:50.906213 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:50.906213 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:50.906213 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:50.907088 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:50.907088 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:50.907088 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:50.907088 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:50.907088 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:05:50.907088 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:05:50.907088 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:05:50.907088 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:05:50.907088 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:05:50.907088 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:05:50.907088 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:05:50.907088 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:05:50.907088 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 88/189] 10.222.0.71 () {56 vars in 1114 bytes} [Mon Jul 4 09:05:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:05:50.908452 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ab4c6-bd1c-4a0c-88ba-f7dbb0979420 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:50.908452 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ab4c6-bd1c-4a0c-88ba-f7dbb0979420 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:50.908452 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ab4c6-bd1c-4a0c-88ba-f7dbb0979420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:50.927497 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-fc1ab4c6-bd1c-4a0c-88ba-f7dbb0979420 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:05:50.928752 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-fc1ab4c6-bd1c-4a0c-88ba-f7dbb0979420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:05:50.956351 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1293ce75-2e15-4038-aa45-17d7a6876468 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:51.000760 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0998006c-6f6c-4160-95c3-3a9c42d27eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:51.001814 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0998006c-6f6c-4160-95c3-3a9c42d27eeb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:51.002387 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0998006c-6f6c-4160-95c3-3a9c42d27eeb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:51.002387 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0998006c-6f6c-4160-95c3-3a9c42d27eeb None admin] PATH_INFO: `/v3/users/1837335e130248a39830dc4da680f52c` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:51.007770 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 89/190] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:05:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:05:51.009743 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2095200-8ab3-4115-8d9e-f85f63a9a8d1 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:51.010534 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2095200-8ab3-4115-8d9e-f85f63a9a8d1 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:51.010812 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2095200-8ab3-4115-8d9e-f85f63a9a8d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:51.018492 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0998006c-6f6c-4160-95c3-3a9c42d27eeb None admin] RBAC: Authorizing `identity:get_user(user_id=1837335e130248a39830dc4da680f52c)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:51.019986 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0998006c-6f6c-4160-95c3-3a9c42d27eeb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:51.021976 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 102/191] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 4 09:05:50 2022] GET /identity/v3/users/1837335e130248a39830dc4da680f52c => generated 283 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:51.022572 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0998006c-6f6c-4160-95c3-3a9c42d27eeb None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:51.031396 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-e2095200-8ab3-4115-8d9e-f85f63a9a8d1 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:05:51.032585 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e2095200-8ab3-4115-8d9e-f85f63a9a8d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:05:51.062975 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:51.104333 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 90/192] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:05:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:05:51.105212 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2095200-8ab3-4115-8d9e-f85f63a9a8d1 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:51.130185 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0998006c-6f6c-4160-95c3-3a9c42d27eeb None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:51.148892 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:51.166690 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a01578e-4819-4ec9-9607-0f608e75a5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:51.167368 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a01578e-4819-4ec9-9607-0f608e75a5c2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:51.167656 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a01578e-4819-4ec9-9607-0f608e75a5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:51.167943 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a01578e-4819-4ec9-9607-0f608e75a5c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:51.168692 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a01578e-4819-4ec9-9607-0f608e75a5c2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:51.169009 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a01578e-4819-4ec9-9607-0f608e75a5c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:51.170411 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a01578e-4819-4ec9-9607-0f608e75a5c2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:51.178224 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 103/193] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 4 09:05:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:51.179023 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a01578e-4819-4ec9-9607-0f608e75a5c2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:51.210393 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2095200-8ab3-4115-8d9e-f85f63a9a8d1 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:51.244351 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb192fee-4c8e-47f9-b90d-ceee70e0aea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:51.245006 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb192fee-4c8e-47f9-b90d-ceee70e0aea8 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:51.245326 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb192fee-4c8e-47f9-b90d-ceee70e0aea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:51.245609 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb192fee-4c8e-47f9-b90d-ceee70e0aea8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:51.280382 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:51.280539 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a01578e-4819-4ec9-9607-0f608e75a5c2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:51.314338 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1f31f4-4534-46f3-886a-3f7995347ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:51.314680 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f31f4-4534-46f3-886a-3f7995347ed0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:51.314969 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f31f4-4534-46f3-886a-3f7995347ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:51.315253 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f31f4-4534-46f3-886a-3f7995347ed0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:51.316122 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb192fee-4c8e-47f9-b90d-ceee70e0aea8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:51.316444 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1f31f4-4534-46f3-886a-3f7995347ed0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:51.317631 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb192fee-4c8e-47f9-b90d-ceee70e0aea8 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:51.317904 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1f31f4-4534-46f3-886a-3f7995347ed0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:51.319523 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 104/194] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:05:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:51.324982 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1f31f4-4534-46f3-886a-3f7995347ed0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:51.368778 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:51.425057 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1f31f4-4534-46f3-886a-3f7995347ed0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:51.429970 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 91/195] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:05:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:05:51.430811 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb192fee-4c8e-47f9-b90d-ceee70e0aea8 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:51.461943 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d3395e-9105-4bda-8e67-eff3343966b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:51.463296 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3395e-9105-4bda-8e67-eff3343966b5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:51.463760 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3395e-9105-4bda-8e67-eff3343966b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:51.464045 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3395e-9105-4bda-8e67-eff3343966b5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:51.473887 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d3395e-9105-4bda-8e67-eff3343966b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:05:51.482380 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-16d3395e-9105-4bda-8e67-eff3343966b5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 09:05:51.482380 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:51.482380 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:51.482380 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:51.482380 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:51.482380 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:51.482380 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:51.482380 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:51.482380 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:51.482380 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:51.482380 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:51.482380 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:51.482380 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:05:51.482380 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:05:51.482380 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:05:51.482380 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:05:51.483330 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:05:51.483330 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:05:51.483330 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:05:51.483330 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:05:51.483330 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:51.483330 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:51.483330 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:51.483330 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:51.483330 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:51.483330 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:51.483330 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:51.483330 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:51.483330 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:51.483330 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:51.483330 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:51.483330 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:51.484139 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:51.484139 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:51.484139 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:51.484139 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:51.484139 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:05:51.484139 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:05:51.484139 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:05:51.484139 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:05:51.484139 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:05:51.484139 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:05:51.484139 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 09:05:51.484139 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:05:51.484139 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 105/196] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:05:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:05:51.484797 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d3395e-9105-4bda-8e67-eff3343966b5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:51.499008 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:51.533022 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb192fee-4c8e-47f9-b90d-ceee70e0aea8 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:51.566920 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ffba48-a4bc-4ece-995d-b47bdccbffcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:51.567725 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ffba48-a4bc-4ece-995d-b47bdccbffcb None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:51.568011 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ffba48-a4bc-4ece-995d-b47bdccbffcb None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:51.568290 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ffba48-a4bc-4ece-995d-b47bdccbffcb None admin] PATH_INFO: `/v3/users/1837335e130248a39830dc4da680f52c` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:51.569536 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ffba48-a4bc-4ece-995d-b47bdccbffcb None admin] RBAC: Authorizing `identity:get_user(user_id=1837335e130248a39830dc4da680f52c)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:51.570990 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ffba48-a4bc-4ece-995d-b47bdccbffcb None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:51.572840 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 92/197] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 4 09:05:51 2022] GET /identity/v3/users/1837335e130248a39830dc4da680f52c => generated 283 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:51.573467 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ffba48-a4bc-4ece-995d-b47bdccbffcb None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:51.597393 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d3395e-9105-4bda-8e67-eff3343966b5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:51.606978 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:51.631957 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a84c60d-4074-4feb-889b-e80cf476c22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:51.632632 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a84c60d-4074-4feb-889b-e80cf476c22a None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:51.632913 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a84c60d-4074-4feb-889b-e80cf476c22a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:51.633228 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a84c60d-4074-4feb-889b-e80cf476c22a None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:51.633940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a84c60d-4074-4feb-889b-e80cf476c22a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:51.635462 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a84c60d-4074-4feb-889b-e80cf476c22a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:51.656661 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 106/198] 10.222.0.71 () {60 vars in 1132 bytes} [Mon Jul 4 09:05:51 2022] POST /identity/v3/users => generated 297 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:05:51.657691 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a84c60d-4074-4feb-889b-e80cf476c22a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:51.693285 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ffba48-a4bc-4ece-995d-b47bdccbffcb None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:51.738124 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b5865b-682b-4dde-b3df-e5e82eea539e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:51.739558 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b5865b-682b-4dde-b3df-e5e82eea539e None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:51.740044 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b5865b-682b-4dde-b3df-e5e82eea539e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:51.740501 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b5865b-682b-4dde-b3df-e5e82eea539e None admin] PATH_INFO: `/v3/projects/f8a33963b3504bf196b09f575de6e329` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:51.742521 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b5865b-682b-4dde-b3df-e5e82eea539e None admin] RBAC: Authorizing `identity:get_project(project_id=f8a33963b3504bf196b09f575de6e329)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:51.745024 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b5865b-682b-4dde-b3df-e5e82eea539e None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:51.753088 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 93/199] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 4 09:05:51 2022] GET /identity/v3/projects/f8a33963b3504bf196b09f575de6e329 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:51.754421 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b5865b-682b-4dde-b3df-e5e82eea539e None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:51.757923 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:51.793024 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a84c60d-4074-4feb-889b-e80cf476c22a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:51.832145 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f101742-6475-4f39-8a17-21a56ec8ab8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:51.832748 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f101742-6475-4f39-8a17-21a56ec8ab8c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:51.833048 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f101742-6475-4f39-8a17-21a56ec8ab8c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:51.833347 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f101742-6475-4f39-8a17-21a56ec8ab8c None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:51.835144 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f101742-6475-4f39-8a17-21a56ec8ab8c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:51.835144 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f101742-6475-4f39-8a17-21a56ec8ab8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:51.835771 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f101742-6475-4f39-8a17-21a56ec8ab8c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:51.846789 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 107/200] 10.222.0.71 () {56 vars in 1101 bytes} [Mon Jul 4 09:05:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:51.847716 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f101742-6475-4f39-8a17-21a56ec8ab8c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:51.877140 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:51.880148 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b5865b-682b-4dde-b3df-e5e82eea539e None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:51.927955 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c955294d-e61c-423f-b382-7d6f4e729059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:51.929308 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c955294d-e61c-423f-b382-7d6f4e729059 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:51.929308 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c955294d-e61c-423f-b382-7d6f4e729059 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:51.929308 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c955294d-e61c-423f-b382-7d6f4e729059 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:51.974582 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f101742-6475-4f39-8a17-21a56ec8ab8c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:52.015245 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c955294d-e61c-423f-b382-7d6f4e729059 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:52.017103 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c955294d-e61c-423f-b382-7d6f4e729059 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:52.017842 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b689f95d-365b-4676-b3ca-28a78e16668b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:52.018509 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b689f95d-365b-4676-b3ca-28a78e16668b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:52.018796 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b689f95d-365b-4676-b3ca-28a78e16668b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:52.019076 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b689f95d-365b-4676-b3ca-28a78e16668b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:52.019949 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b689f95d-365b-4676-b3ca-28a78e16668b None admin] RBAC: Adding query filter params (role.id=f36e053b442540f08041ac7878760f65, scope.project.id=f8a33963b3504bf196b09f575de6e329, user.id=1837335e130248a39830dc4da680f52c) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:52.020261 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b689f95d-365b-4676-b3ca-28a78e16668b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:52.020611 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:52.021722 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b689f95d-365b-4676-b3ca-28a78e16668b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:52.028901 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 108/201] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 4 09:05:51 2022] GET /identity/v3/role_assignments?user.id=1837335e130248a39830dc4da680f52c&scope.project.id=f8a33963b3504bf196b09f575de6e329&role.id=f36e053b442540f08041ac7878760f65 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:52.029840 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b689f95d-365b-4676-b3ca-28a78e16668b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:52.206951 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 94/202] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:05:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:05:52.215106 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c955294d-e61c-423f-b382-7d6f4e729059 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:52.240986 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b689f95d-365b-4676-b3ca-28a78e16668b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:52.243005 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:52.283569 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63fa7da-860a-4cb2-bcd7-1d37e954461e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:52.285617 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63fa7da-860a-4cb2-bcd7-1d37e954461e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:52.286344 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63fa7da-860a-4cb2-bcd7-1d37e954461e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:52.286825 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63fa7da-860a-4cb2-bcd7-1d37e954461e None admin] PATH_INFO: `/v3/users/639cd4e0d5ca4e4d8f6bab272d48a607` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:52.288564 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63fa7da-860a-4cb2-bcd7-1d37e954461e None admin] RBAC: Authorizing `identity:get_user(user_id=639cd4e0d5ca4e4d8f6bab272d48a607)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:52.291908 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63fa7da-860a-4cb2-bcd7-1d37e954461e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:52.294159 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 109/203] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 4 09:05:52 2022] GET /identity/v3/users/639cd4e0d5ca4e4d8f6bab272d48a607 => generated 289 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:52.299659 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63fa7da-860a-4cb2-bcd7-1d37e954461e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:52.403827 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c955294d-e61c-423f-b382-7d6f4e729059 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:52.413372 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:52.426071 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63fa7da-860a-4cb2-bcd7-1d37e954461e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:52.475032 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46350edf-70a7-420f-9985-6a395377a775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:52.476029 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46350edf-70a7-420f-9985-6a395377a775 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:52.476516 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46350edf-70a7-420f-9985-6a395377a775 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:52.477010 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46350edf-70a7-420f-9985-6a395377a775 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:52.477712 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa475d6-b24d-42a6-94a2-8765a22e2e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:52.479144 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa475d6-b24d-42a6-94a2-8765a22e2e0e None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:52.479428 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa475d6-b24d-42a6-94a2-8765a22e2e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:52.479707 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa475d6-b24d-42a6-94a2-8765a22e2e0e None admin] PATH_INFO: `/v3/projects/f8a33963b3504bf196b09f575de6e329` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:52.480941 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa475d6-b24d-42a6-94a2-8765a22e2e0e None admin] RBAC: Authorizing `identity:get_project(project_id=f8a33963b3504bf196b09f575de6e329)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:52.482451 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa475d6-b24d-42a6-94a2-8765a22e2e0e None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:52.488995 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 95/204] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 4 09:05:52 2022] GET /identity/v3/projects/f8a33963b3504bf196b09f575de6e329 => generated 302 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:05:52.494014 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa475d6-b24d-42a6-94a2-8765a22e2e0e None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:52.513402 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:52.555262 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46350edf-70a7-420f-9985-6a395377a775 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:52.557196 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46350edf-70a7-420f-9985-6a395377a775 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:52.660795 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 110/205] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:05:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:05:52.667088 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46350edf-70a7-420f-9985-6a395377a775 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:52.712723 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa475d6-b24d-42a6-94a2-8765a22e2e0e None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:52.762659 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d0e04a-7466-460e-a32c-21f72e51db67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:52.763636 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d0e04a-7466-460e-a32c-21f72e51db67 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:52.764123 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d0e04a-7466-460e-a32c-21f72e51db67 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:52.764601 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d0e04a-7466-460e-a32c-21f72e51db67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:52.765808 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d0e04a-7466-460e-a32c-21f72e51db67 None admin] RBAC: Adding query filter params (role.id=5fb58444d5604bf689014e27dccacb45, scope.project.id=f8a33963b3504bf196b09f575de6e329, user.id=1837335e130248a39830dc4da680f52c) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:52.766354 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d0e04a-7466-460e-a32c-21f72e51db67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:52.768569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d0e04a-7466-460e-a32c-21f72e51db67 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:52.770702 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:52.775531 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 96/206] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 4 09:05:52 2022] GET /identity/v3/role_assignments?user.id=1837335e130248a39830dc4da680f52c&scope.project.id=f8a33963b3504bf196b09f575de6e329&role.id=5fb58444d5604bf689014e27dccacb45 => generated 259 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:52.846853 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46350edf-70a7-420f-9985-6a395377a775 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:52.891684 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0fd68a-8924-40cc-a2f6-06ebc6262de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:52.892569 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0fd68a-8924-40cc-a2f6-06ebc6262de7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:52.892855 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0fd68a-8924-40cc-a2f6-06ebc6262de7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:52.893165 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0fd68a-8924-40cc-a2f6-06ebc6262de7 None admin] PATH_INFO: `/v3/projects/e192ac889bad428985088eea5ce750ab` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:52.894414 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0fd68a-8924-40cc-a2f6-06ebc6262de7 None admin] RBAC: Authorizing `identity:get_project(project_id=e192ac889bad428985088eea5ce750ab)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:52.895890 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0fd68a-8924-40cc-a2f6-06ebc6262de7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:52.897685 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 111/207] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 4 09:05:52 2022] GET /identity/v3/projects/e192ac889bad428985088eea5ce750ab => generated 298 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:52.904259 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d0e04a-7466-460e-a32c-21f72e51db67 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:52.940071 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:53.032945 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d0e04a-7466-460e-a32c-21f72e51db67 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:53.074813 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8600d7-c0fc-4d12-8a8b-a77312d0a14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:53.075486 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8600d7-c0fc-4d12-8a8b-a77312d0a14e None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:53.075821 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8600d7-c0fc-4d12-8a8b-a77312d0a14e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:53.076106 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8600d7-c0fc-4d12-8a8b-a77312d0a14e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:53.076958 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8600d7-c0fc-4d12-8a8b-a77312d0a14e None admin] RBAC: Adding query filter params (role.id=56205a5df6514298a69fde673550f0aa, scope.project.id=e192ac889bad428985088eea5ce750ab, user.id=639cd4e0d5ca4e4d8f6bab272d48a607) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:53.077363 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8600d7-c0fc-4d12-8a8b-a77312d0a14e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:53.078829 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8600d7-c0fc-4d12-8a8b-a77312d0a14e None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:53.090388 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 97/208] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 4 09:05:52 2022] GET /identity/v3/role_assignments?user.id=639cd4e0d5ca4e4d8f6bab272d48a607&scope.project.id=e192ac889bad428985088eea5ce750ab&role.id=56205a5df6514298a69fde673550f0aa => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:54.066934 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32544a4b-67b7-4436-8ad6-950a54b0643d None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:54.067949 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32544a4b-67b7-4436-8ad6-950a54b0643d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:54.068487 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32544a4b-67b7-4436-8ad6-950a54b0643d None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:54.070180 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 112/209] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:05:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:05:54.076526 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec648e-7147-4416-ba23-f759252eca52 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:54.077386 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec648e-7147-4416-ba23-f759252eca52 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:54.077853 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec648e-7147-4416-ba23-f759252eca52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:54.095975 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-dfec648e-7147-4416-ba23-f759252eca52 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:05:54.097534 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-dfec648e-7147-4416-ba23-f759252eca52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:05:54.173539 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 98/210] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:05:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:05:54.201484 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32544a4b-67b7-4436-8ad6-950a54b0643d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:54.276190 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b81532-9d93-4518-9bcd-6f181fc251b1 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:54.276534 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b81532-9d93-4518-9bcd-6f181fc251b1 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:54.276767 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b81532-9d93-4518-9bcd-6f181fc251b1 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:54.278314 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 99/211] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:05:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:05:54.283319 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a21fcfc-9e11-4db9-99de-731330142522 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:54.283619 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a21fcfc-9e11-4db9-99de-731330142522 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:54.283910 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a21fcfc-9e11-4db9-99de-731330142522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:54.301844 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-2a21fcfc-9e11-4db9-99de-731330142522 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:05:54.303206 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-2a21fcfc-9e11-4db9-99de-731330142522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:05:54.311174 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32544a4b-67b7-4436-8ad6-950a54b0643d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:54.329026 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:54.341696 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:54.350931 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d2a0e8-30fe-445e-a22f-55185bb47355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:54.351989 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d2a0e8-30fe-445e-a22f-55185bb47355 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:54.352492 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d2a0e8-30fe-445e-a22f-55185bb47355 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:54.352963 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d2a0e8-30fe-445e-a22f-55185bb47355 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:54.354472 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d2a0e8-30fe-445e-a22f-55185bb47355 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:54.356276 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d2a0e8-30fe-445e-a22f-55185bb47355 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:54.358236 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 113/212] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:05:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:54.363590 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d2a0e8-30fe-445e-a22f-55185bb47355 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:54.406453 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 100/213] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:05:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:05:54.415789 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64938219-b6b9-4fa1-ac62-e75edcbf0709 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:54.417238 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64938219-b6b9-4fa1-ac62-e75edcbf0709 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:54.417238 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64938219-b6b9-4fa1-ac62-e75edcbf0709 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:54.419557 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 101/214] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:05:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:05:54.424878 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb555027-1481-4350-90fe-9eaf1ca4a268 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:54.425407 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb555027-1481-4350-90fe-9eaf1ca4a268 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:54.425952 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb555027-1481-4350-90fe-9eaf1ca4a268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:54.444373 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-bb555027-1481-4350-90fe-9eaf1ca4a268 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:05:54.445599 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-bb555027-1481-4350-90fe-9eaf1ca4a268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:05:54.496777 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d2a0e8-30fe-445e-a22f-55185bb47355 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:54.516332 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 102/215] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:05:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:05:54.522065 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb555027-1481-4350-90fe-9eaf1ca4a268 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:54.539402 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8300416-8468-4aad-9e42-af318a262d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:54.540097 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8300416-8468-4aad-9e42-af318a262d5e None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:54.540387 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8300416-8468-4aad-9e42-af318a262d5e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:54.540670 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8300416-8468-4aad-9e42-af318a262d5e None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:54.541485 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8300416-8468-4aad-9e42-af318a262d5e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:54.542945 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8300416-8468-4aad-9e42-af318a262d5e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:54.565415 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 114/216] 10.222.0.71 () {60 vars in 1132 bytes} [Mon Jul 4 09:05:54 2022] POST /identity/v3/users => generated 297 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:05:54.574304 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8300416-8468-4aad-9e42-af318a262d5e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:54.639211 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:54.645903 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:54.658148 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb555027-1481-4350-90fe-9eaf1ca4a268 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:54.708966 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8300416-8468-4aad-9e42-af318a262d5e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:54.709196 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f7b0e8-02b2-42ca-9105-2b099a07a39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:54.710002 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f7b0e8-02b2-42ca-9105-2b099a07a39e None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:54.710286 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f7b0e8-02b2-42ca-9105-2b099a07a39e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:54.710565 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f7b0e8-02b2-42ca-9105-2b099a07a39e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:54.726835 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f7b0e8-02b2-42ca-9105-2b099a07a39e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:05:54.735916 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-c4f7b0e8-02b2-42ca-9105-2b099a07a39e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:05:54.735916 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:54.735916 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:54.735916 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:54.735916 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:54.735916 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:54.735916 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:54.735916 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:54.735916 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:54.735916 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:54.735916 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:54.735916 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:54.735916 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:05:54.735916 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:05:54.735916 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:05:54.735916 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:05:54.736888 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:05:54.736888 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:05:54.736888 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:05:54.736888 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:05:54.736888 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:54.736888 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:54.736888 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:54.736888 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:54.736888 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:54.736888 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:54.736888 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:54.736888 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:54.736888 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:54.736888 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:54.736888 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:54.736888 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:54.737776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:54.737776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:54.737776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:54.737776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:54.737776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:05:54.737776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:05:54.737776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:05:54.737776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:05:54.737776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:05:54.737776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:05:54.737776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:05:54.737776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:05:54.737776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 103/217] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:05:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:05:54.746090 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f7b0e8-02b2-42ca-9105-2b099a07a39e None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:54.762113 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dfd5f0-5cbc-450e-8ece-7deacd0e3a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:54.762697 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dfd5f0-5cbc-450e-8ece-7deacd0e3a95 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:54.763039 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dfd5f0-5cbc-450e-8ece-7deacd0e3a95 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:54.763827 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dfd5f0-5cbc-450e-8ece-7deacd0e3a95 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:54.775027 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dfd5f0-5cbc-450e-8ece-7deacd0e3a95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:05:54.783581 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-a8dfd5f0-5cbc-450e-8ece-7deacd0e3a95 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:05:54.783581 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:54.783581 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:54.783581 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:54.783581 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:54.783581 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:54.783581 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:54.783581 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:54.783581 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:54.783581 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:54.783581 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:54.783581 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:54.783581 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:05:54.783581 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:05:54.783581 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:05:54.783581 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:05:54.784496 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:05:54.784496 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:05:54.784496 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:05:54.784496 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:05:54.784496 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:54.784496 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:54.784496 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:54.784496 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:54.784496 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:54.784496 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:54.784496 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:54.784496 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:54.784496 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:54.784496 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:54.784496 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:54.784496 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:54.785336 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:54.785336 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:54.785336 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:54.785336 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:54.785336 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:05:54.785336 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:05:54.785336 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:05:54.785336 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:05:54.785336 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:05:54.785336 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:05:54.785336 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:05:54.785336 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:05:54.786512 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 115/218] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 4 09:05:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:05:54.793199 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dfd5f0-5cbc-450e-8ece-7deacd0e3a95 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:54.888010 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f7b0e8-02b2-42ca-9105-2b099a07a39e None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:54.932983 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dfd5f0-5cbc-450e-8ece-7deacd0e3a95 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:54.943508 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db79651d-ff94-4e6a-aeb6-8c380865120b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:54.944200 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79651d-ff94-4e6a-aeb6-8c380865120b None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:54.944487 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79651d-ff94-4e6a-aeb6-8c380865120b None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:54.944767 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79651d-ff94-4e6a-aeb6-8c380865120b None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:54.945904 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db79651d-ff94-4e6a-aeb6-8c380865120b None admin] RBAC: Adding query filter params (name=member) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:54.946371 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db79651d-ff94-4e6a-aeb6-8c380865120b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:54.946551 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:54.948544 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db79651d-ff94-4e6a-aeb6-8c380865120b None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:54.960463 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 104/219] 10.222.0.71 () {56 vars in 1101 bytes} [Mon Jul 4 09:05:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:54.966320 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db79651d-ff94-4e6a-aeb6-8c380865120b None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:54.974553 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1f4563-1819-40dd-b0b5-fc6b57557b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:54.975332 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f4563-1819-40dd-b0b5-fc6b57557b37 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:54.975621 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f4563-1819-40dd-b0b5-fc6b57557b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:54.975900 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f4563-1819-40dd-b0b5-fc6b57557b37 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:54.976664 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1f4563-1819-40dd-b0b5-fc6b57557b37 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:54.976981 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1f4563-1819-40dd-b0b5-fc6b57557b37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:54.979101 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1f4563-1819-40dd-b0b5-fc6b57557b37 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:54.989420 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:54.990637 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 116/220] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:05:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:05:54.998467 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1f4563-1819-40dd-b0b5-fc6b57557b37 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:55.088780 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db79651d-ff94-4e6a-aeb6-8c380865120b None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:55.126601 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1086fdf0-e861-4d34-bc57-d8b779a71ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:55.127680 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1086fdf0-e861-4d34-bc57-d8b779a71ee5 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:55.128042 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1086fdf0-e861-4d34-bc57-d8b779a71ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:55.128402 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1086fdf0-e861-4d34-bc57-d8b779a71ee5 None admin] PATH_INFO: `/v3/users/abd9f7d1a38340afa59c58abfabbf5f5` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:55.129860 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1f4563-1819-40dd-b0b5-fc6b57557b37 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:55.144078 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1086fdf0-e861-4d34-bc57-d8b779a71ee5 None admin] RBAC: Authorizing `identity:get_user(user_id=abd9f7d1a38340afa59c58abfabbf5f5)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:55.145745 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1086fdf0-e861-4d34-bc57-d8b779a71ee5 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:55.147851 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 105/221] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 4 09:05:54 2022] GET /identity/v3/users/abd9f7d1a38340afa59c58abfabbf5f5 => generated 297 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:55.149375 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86294d-5afe-4244-842d-f4a2594c015c None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:55.149833 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86294d-5afe-4244-842d-f4a2594c015c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:55.151117 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86294d-5afe-4244-842d-f4a2594c015c None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:55.152715 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 106/222] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:05:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:05:55.159918 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ca091-aeb5-443c-bd71-9a9a4b2eef6a None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:55.160207 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ca091-aeb5-443c-bd71-9a9a4b2eef6a None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:55.160488 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ca091-aeb5-443c-bd71-9a9a4b2eef6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:55.172060 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a827b6-a55d-4065-97df-6155bb7269d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:55.173316 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a827b6-a55d-4065-97df-6155bb7269d3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:55.173803 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a827b6-a55d-4065-97df-6155bb7269d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:55.174254 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a827b6-a55d-4065-97df-6155bb7269d3 None admin] PATH_INFO: `/v3/users/1837335e130248a39830dc4da680f52c` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:55.175684 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a827b6-a55d-4065-97df-6155bb7269d3 None admin] RBAC: Authorizing `identity:get_user(user_id=1837335e130248a39830dc4da680f52c)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:55.177363 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a827b6-a55d-4065-97df-6155bb7269d3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:55.178409 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-c41ca091-aeb5-443c-bd71-9a9a4b2eef6a None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:05:55.179280 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 117/223] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 4 09:05:54 2022] GET /identity/v3/users/1837335e130248a39830dc4da680f52c => generated 283 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:55.179947 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c41ca091-aeb5-443c-bd71-9a9a4b2eef6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:05:55.180247 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a827b6-a55d-4065-97df-6155bb7269d3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:55.226736 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:55.251989 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:55.258710 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 107/224] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:05:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:05:55.260021 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ca091-aeb5-443c-bd71-9a9a4b2eef6a None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:55.315521 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a827b6-a55d-4065-97df-6155bb7269d3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:55.351868 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c195da2-901b-4c85-8b5a-0b10a2cc3d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:55.352584 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c195da2-901b-4c85-8b5a-0b10a2cc3d97 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:55.352876 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c195da2-901b-4c85-8b5a-0b10a2cc3d97 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:55.353213 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c195da2-901b-4c85-8b5a-0b10a2cc3d97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:55.368262 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ca091-aeb5-443c-bd71-9a9a4b2eef6a None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:55.414723 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0d6ac6-dd89-49e7-b7e0-0b8e51747841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:55.416425 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0d6ac6-dd89-49e7-b7e0-0b8e51747841 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:55.417092 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0d6ac6-dd89-49e7-b7e0-0b8e51747841 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:55.417624 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0d6ac6-dd89-49e7-b7e0-0b8e51747841 None admin] PATH_INFO: `/v3/projects/f8a33963b3504bf196b09f575de6e329` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:55.419022 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0d6ac6-dd89-49e7-b7e0-0b8e51747841 None admin] RBAC: Authorizing `identity:get_project(project_id=f8a33963b3504bf196b09f575de6e329)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:55.420678 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0d6ac6-dd89-49e7-b7e0-0b8e51747841 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:55.422620 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 108/225] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 4 09:05:55 2022] GET /identity/v3/projects/f8a33963b3504bf196b09f575de6e329 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:55.423604 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0d6ac6-dd89-49e7-b7e0-0b8e51747841 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:55.426144 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c195da2-901b-4c85-8b5a-0b10a2cc3d97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:55.428073 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c195da2-901b-4c85-8b5a-0b10a2cc3d97 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:55.471894 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:55.492710 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:55.531488 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0d6ac6-dd89-49e7-b7e0-0b8e51747841 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:55.538846 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 118/226] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:05:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:05:55.539681 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c195da2-901b-4c85-8b5a-0b10a2cc3d97 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:55.572550 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0f91b6-9069-4c76-af69-63b6202e2c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:55.573416 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f91b6-9069-4c76-af69-63b6202e2c47 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:55.573767 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f91b6-9069-4c76-af69-63b6202e2c47 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:55.574212 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f91b6-9069-4c76-af69-63b6202e2c47 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:55.585016 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0f91b6-9069-4c76-af69-63b6202e2c47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:05:55.594163 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-9f0f91b6-9069-4c76-af69-63b6202e2c47 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:05:55.594163 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:55.594163 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:55.594163 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:55.594163 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:55.594163 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:55.594163 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:55.594163 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:55.594163 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:55.594163 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:55.594163 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:55.594163 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:55.594163 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:05:55.594163 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:05:55.594163 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:05:55.594163 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:05:55.595079 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:05:55.595079 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:05:55.595079 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:05:55.595079 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:05:55.595079 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:55.595079 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:55.595079 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:55.595079 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:55.595079 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:55.595079 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:55.595079 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:55.595079 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:55.595079 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:55.595079 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:55.595079 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:55.595079 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:55.595888 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:55.595888 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:55.595888 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:55.595888 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:55.595888 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:05:55.595888 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:05:55.595888 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:05:55.595888 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:05:55.595888 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:05:55.595888 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:05:55.595888 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:05:55.595888 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:05:55.595888 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 109/227] 10.222.0.71 () {56 vars in 1114 bytes} [Mon Jul 4 09:05:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:05:55.601307 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0f91b6-9069-4c76-af69-63b6202e2c47 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:55.653063 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c195da2-901b-4c85-8b5a-0b10a2cc3d97 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:55.692327 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1497bf-a442-4aa1-bee5-8ddb910c1768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:55.693267 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1497bf-a442-4aa1-bee5-8ddb910c1768 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:55.693554 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1497bf-a442-4aa1-bee5-8ddb910c1768 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:55.693837 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1497bf-a442-4aa1-bee5-8ddb910c1768 None admin] PATH_INFO: `/v3/projects/f8a33963b3504bf196b09f575de6e329/users/1837335e130248a39830dc4da680f52c/roles/f36e053b442540f08041ac7878760f65` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:55.695937 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1497bf-a442-4aa1-bee5-8ddb910c1768 None admin] RBAC: Authorizing `identity:create_grant(project_id=f8a33963b3504bf196b09f575de6e329, user_id=1837335e130248a39830dc4da680f52c, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:55.698244 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1497bf-a442-4aa1-bee5-8ddb910c1768 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:55.708374 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0f91b6-9069-4c76-af69-63b6202e2c47 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:55.708920 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:55.717084 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 119/228] 10.222.0.71 () {58 vars in 1439 bytes} [Mon Jul 4 09:05:55 2022] PUT /identity/v3/projects/f8a33963b3504bf196b09f575de6e329/users/1837335e130248a39830dc4da680f52c/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:05:55.726667 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1497bf-a442-4aa1-bee5-8ddb910c1768 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:55.737931 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:55.749747 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3360fa85-75ff-4a9d-9e28-c0fec656e227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:55.750539 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3360fa85-75ff-4a9d-9e28-c0fec656e227 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:55.750825 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3360fa85-75ff-4a9d-9e28-c0fec656e227 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:55.751104 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3360fa85-75ff-4a9d-9e28-c0fec656e227 None admin] PATH_INFO: `/v3/projects/f8a33963b3504bf196b09f575de6e329` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:55.754744 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3360fa85-75ff-4a9d-9e28-c0fec656e227 None admin] RBAC: Authorizing `identity:get_project(project_id=f8a33963b3504bf196b09f575de6e329)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:55.756700 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3360fa85-75ff-4a9d-9e28-c0fec656e227 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:55.765458 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 110/229] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 4 09:05:55 2022] GET /identity/v3/projects/f8a33963b3504bf196b09f575de6e329 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:55.766750 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3360fa85-75ff-4a9d-9e28-c0fec656e227 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:55.908784 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1497bf-a442-4aa1-bee5-8ddb910c1768 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:55.946707 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654511d9-d983-4c08-b2f4-c31afaa24b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:55.947449 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3360fa85-75ff-4a9d-9e28-c0fec656e227 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:55.948600 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654511d9-d983-4c08-b2f4-c31afaa24b2e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:55.948888 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654511d9-d983-4c08-b2f4-c31afaa24b2e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:55.949199 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654511d9-d983-4c08-b2f4-c31afaa24b2e None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:55.949964 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654511d9-d983-4c08-b2f4-c31afaa24b2e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:55.950271 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654511d9-d983-4c08-b2f4-c31afaa24b2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:55.951665 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654511d9-d983-4c08-b2f4-c31afaa24b2e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:55.960505 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 120/230] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 4 09:05:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:55.967525 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654511d9-d983-4c08-b2f4-c31afaa24b2e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:55.985638 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af08e2cd-f332-447c-bc44-70360d5a454a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:55.986314 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af08e2cd-f332-447c-bc44-70360d5a454a None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:55.986703 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af08e2cd-f332-447c-bc44-70360d5a454a None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:55.986992 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af08e2cd-f332-447c-bc44-70360d5a454a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:55.987977 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af08e2cd-f332-447c-bc44-70360d5a454a None admin] RBAC: Adding query filter params (role.id=8692e99e9463459f9b7caa96b3051b93, scope.project.id=f8a33963b3504bf196b09f575de6e329, user.id=abd9f7d1a38340afa59c58abfabbf5f5) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:55.988269 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af08e2cd-f332-447c-bc44-70360d5a454a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:55.989752 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af08e2cd-f332-447c-bc44-70360d5a454a None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:55.995819 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 111/231] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 4 09:05:55 2022] GET /identity/v3/role_assignments?user.id=abd9f7d1a38340afa59c58abfabbf5f5&scope.project.id=f8a33963b3504bf196b09f575de6e329&role.id=8692e99e9463459f9b7caa96b3051b93 => generated 259 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:56.053170 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:56.111814 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654511d9-d983-4c08-b2f4-c31afaa24b2e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:56.151851 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d85a4af-1a2c-40be-b10f-0df88c90e930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:56.152748 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d85a4af-1a2c-40be-b10f-0df88c90e930 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:56.153030 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d85a4af-1a2c-40be-b10f-0df88c90e930 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:56.153338 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d85a4af-1a2c-40be-b10f-0df88c90e930 None admin] PATH_INFO: `/v3/users/1837335e130248a39830dc4da680f52c` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:56.154625 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d85a4af-1a2c-40be-b10f-0df88c90e930 None admin] RBAC: Authorizing `identity:get_user(user_id=1837335e130248a39830dc4da680f52c)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:56.156101 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d85a4af-1a2c-40be-b10f-0df88c90e930 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:56.158147 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 121/232] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 4 09:05:55 2022] GET /identity/v3/users/1837335e130248a39830dc4da680f52c => generated 283 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:56.163919 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af08e2cd-f332-447c-bc44-70360d5a454a None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:56.245773 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:56.289802 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af08e2cd-f332-447c-bc44-70360d5a454a None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:56.336167 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01b5530-c5b2-4f48-94fc-4ff5d9921788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:56.336957 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b5530-c5b2-4f48-94fc-4ff5d9921788 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:56.338017 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b5530-c5b2-4f48-94fc-4ff5d9921788 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:56.338941 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b5530-c5b2-4f48-94fc-4ff5d9921788 None admin] PATH_INFO: `/v3/projects/f8a33963b3504bf196b09f575de6e329` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:56.344939 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01b5530-c5b2-4f48-94fc-4ff5d9921788 None admin] RBAC: Authorizing `identity:get_project(project_id=f8a33963b3504bf196b09f575de6e329)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:56.346458 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01b5530-c5b2-4f48-94fc-4ff5d9921788 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:56.348474 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 112/233] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 4 09:05:56 2022] GET /identity/v3/projects/f8a33963b3504bf196b09f575de6e329 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:56.355285 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d85a4af-1a2c-40be-b10f-0df88c90e930 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:56.548951 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d85a4af-1a2c-40be-b10f-0df88c90e930 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:56.583239 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:56.587018 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f8efc8-5a6a-4bf7-b6f2-252ac23a2ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:56.589308 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8efc8-5a6a-4bf7-b6f2-252ac23a2ef6 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:56.589593 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8efc8-5a6a-4bf7-b6f2-252ac23a2ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:56.589871 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8efc8-5a6a-4bf7-b6f2-252ac23a2ef6 None admin] PATH_INFO: `/v3/projects/f8a33963b3504bf196b09f575de6e329/users/1837335e130248a39830dc4da680f52c/roles/5fb58444d5604bf689014e27dccacb45` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:56.592633 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f8efc8-5a6a-4bf7-b6f2-252ac23a2ef6 None admin] RBAC: Authorizing `identity:create_grant(project_id=f8a33963b3504bf196b09f575de6e329, user_id=1837335e130248a39830dc4da680f52c, role_id=5fb58444d5604bf689014e27dccacb45)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:56.594157 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f8efc8-5a6a-4bf7-b6f2-252ac23a2ef6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:56.643793 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 122/234] 10.222.0.71 () {58 vars in 1439 bytes} [Mon Jul 4 09:05:56 2022] PUT /identity/v3/projects/f8a33963b3504bf196b09f575de6e329/users/1837335e130248a39830dc4da680f52c/roles/5fb58444d5604bf689014e27dccacb45 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:05:57.083273 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f3e1d-b55f-4a93-8de7-76c953a61948 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:57.083667 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f3e1d-b55f-4a93-8de7-76c953a61948 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:57.083862 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f3e1d-b55f-4a93-8de7-76c953a61948 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:57.085348 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 113/235] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:05:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:05:57.091064 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b377dc59-fa41-436f-b130-1c1cdcd48f36 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:57.091365 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b377dc59-fa41-436f-b130-1c1cdcd48f36 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:57.091652 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b377dc59-fa41-436f-b130-1c1cdcd48f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:57.111625 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b377dc59-fa41-436f-b130-1c1cdcd48f36 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:05:57.112823 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b377dc59-fa41-436f-b130-1c1cdcd48f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:05:57.215572 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 123/236] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:05:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:05:57.252476 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f3e1d-b55f-4a93-8de7-76c953a61948 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:57.364613 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8d798-6bf0-4148-a4c0-632db4d9ee11 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:57.364801 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8d798-6bf0-4148-a4c0-632db4d9ee11 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:57.365106 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8d798-6bf0-4148-a4c0-632db4d9ee11 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:57.366504 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 124/237] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:05:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:05:57.379705 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3780079c-eaff-41b1-8bc0-c521d8b4b945 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:57.380003 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3780079c-eaff-41b1-8bc0-c521d8b4b945 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:57.380292 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3780079c-eaff-41b1-8bc0-c521d8b4b945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:57.383488 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f3e1d-b55f-4a93-8de7-76c953a61948 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:57.413500 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:57.426988 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21190434-3615-495c-b698-ea4b7b2c8f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:57.427845 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21190434-3615-495c-b698-ea4b7b2c8f90 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:57.428139 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21190434-3615-495c-b698-ea4b7b2c8f90 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:57.428420 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21190434-3615-495c-b698-ea4b7b2c8f90 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:57.429819 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21190434-3615-495c-b698-ea4b7b2c8f90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:57.433903 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21190434-3615-495c-b698-ea4b7b2c8f90 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:57.437692 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3780079c-eaff-41b1-8bc0-c521d8b4b945 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:05:57.438318 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 114/238] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:05:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:57.439263 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3780079c-eaff-41b1-8bc0-c521d8b4b945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:05:57.442480 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21190434-3615-495c-b698-ea4b7b2c8f90 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:57.563092 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 125/239] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:05:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:05:57.589243 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3780079c-eaff-41b1-8bc0-c521d8b4b945 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:57.616295 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21190434-3615-495c-b698-ea4b7b2c8f90 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:57.651770 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d30758-69ec-4adc-abea-532e35d9674a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:57.652519 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d30758-69ec-4adc-abea-532e35d9674a None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:57.652812 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d30758-69ec-4adc-abea-532e35d9674a None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:57.653127 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d30758-69ec-4adc-abea-532e35d9674a None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:57.654154 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d30758-69ec-4adc-abea-532e35d9674a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:57.655630 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d30758-69ec-4adc-abea-532e35d9674a None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:57.690276 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 115/240] 10.222.0.71 () {60 vars in 1132 bytes} [Mon Jul 4 09:05:57 2022] POST /identity/v3/users => generated 293 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:05:57.718618 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:57.785820 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3780079c-eaff-41b1-8bc0-c521d8b4b945 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:57.825338 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713a394c-cd42-48b0-8ecb-64e60885740f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:57.826168 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713a394c-cd42-48b0-8ecb-64e60885740f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:57.826454 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713a394c-cd42-48b0-8ecb-64e60885740f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:57.826730 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713a394c-cd42-48b0-8ecb-64e60885740f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:57.839065 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713a394c-cd42-48b0-8ecb-64e60885740f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:05:57.847058 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-713a394c-cd42-48b0-8ecb-64e60885740f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 09:05:57.847058 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:57.847058 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:57.847058 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:57.847058 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:57.847058 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:57.847058 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:57.847058 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:57.847058 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:57.847058 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:57.847058 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:57.847058 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:57.847058 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:05:57.847058 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:05:57.847058 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:05:57.847058 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:05:57.848007 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:05:57.848007 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:05:57.848007 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:05:57.848007 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:05:57.848007 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:57.848007 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:57.848007 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:57.848007 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:57.848007 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:57.848007 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:57.848007 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:57.848007 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:57.848007 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:57.848007 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:57.848007 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:57.848007 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:57.848821 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:57.848821 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:57.848821 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:57.848821 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:57.848821 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:05:57.848821 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:05:57.848821 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:05:57.848821 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:05:57.848821 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:05:57.848821 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:05:57.848821 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 09:05:57.848821 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:05:57.848821 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 126/241] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:05:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:05:57.854964 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d30758-69ec-4adc-abea-532e35d9674a None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:57.921188 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:57.977467 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d30758-69ec-4adc-abea-532e35d9674a None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:58.020827 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b06d75-1419-401f-bde6-796772bb0004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:58.021598 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b06d75-1419-401f-bde6-796772bb0004 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:58.021893 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b06d75-1419-401f-bde6-796772bb0004 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:58.022177 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b06d75-1419-401f-bde6-796772bb0004 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:58.022949 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b06d75-1419-401f-bde6-796772bb0004 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:58.023263 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b06d75-1419-401f-bde6-796772bb0004 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:58.024682 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b06d75-1419-401f-bde6-796772bb0004 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:58.033763 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 116/242] 10.222.0.71 () {56 vars in 1101 bytes} [Mon Jul 4 09:05:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:58.039541 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713a394c-cd42-48b0-8ecb-64e60885740f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:58.174397 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713a394c-cd42-48b0-8ecb-64e60885740f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:58.187477 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:58.218847 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d6aff9-7708-4acf-af7b-0142afe91641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:58.219796 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d6aff9-7708-4acf-af7b-0142afe91641 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:58.220087 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d6aff9-7708-4acf-af7b-0142afe91641 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:58.220368 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d6aff9-7708-4acf-af7b-0142afe91641 None admin] PATH_INFO: `/v3/users/58287e2295c04e5f89626f1fc39bfb5d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:58.237850 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d6aff9-7708-4acf-af7b-0142afe91641 None admin] RBAC: Authorizing `identity:get_user(user_id=58287e2295c04e5f89626f1fc39bfb5d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:58.239341 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d6aff9-7708-4acf-af7b-0142afe91641 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:58.241459 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 127/243] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 4 09:05:58 2022] GET /identity/v3/users/58287e2295c04e5f89626f1fc39bfb5d => generated 297 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:58.246650 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b06d75-1419-401f-bde6-796772bb0004 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:58.361753 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b06d75-1419-401f-bde6-796772bb0004 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:58.376146 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:58.400879 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258d03ba-27bc-4621-916e-5aeed4911971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:58.401670 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258d03ba-27bc-4621-916e-5aeed4911971 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:58.401960 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258d03ba-27bc-4621-916e-5aeed4911971 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:58.402246 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258d03ba-27bc-4621-916e-5aeed4911971 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:58.475458 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258d03ba-27bc-4621-916e-5aeed4911971 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:58.476948 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258d03ba-27bc-4621-916e-5aeed4911971 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:58.504702 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc590c11-d24d-4b86-8cdc-9dccd72fbb49 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:58.505010 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc590c11-d24d-4b86-8cdc-9dccd72fbb49 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:58.505305 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc590c11-d24d-4b86-8cdc-9dccd72fbb49 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:58.506696 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 128/244] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:05:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:05:58.512833 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c16d05-1eb8-4e0a-a605-b4bb62630ff7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:58.513150 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c16d05-1eb8-4e0a-a605-b4bb62630ff7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:58.513437 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c16d05-1eb8-4e0a-a605-b4bb62630ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:58.533175 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-89c16d05-1eb8-4e0a-a605-b4bb62630ff7 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:05:58.534493 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-89c16d05-1eb8-4e0a-a605-b4bb62630ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:05:58.594778 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 117/245] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:05:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:05:58.601176 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258d03ba-27bc-4621-916e-5aeed4911971 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:58.614796 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 129/246] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:05:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:05:58.616331 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:58.644466 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c16d05-1eb8-4e0a-a605-b4bb62630ff7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:58.710661 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258d03ba-27bc-4621-916e-5aeed4911971 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:58.747833 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6804d39-40e0-41b4-af6a-ca8606a4e89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:58.748639 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6804d39-40e0-41b4-af6a-ca8606a4e89e None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:58.748927 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6804d39-40e0-41b4-af6a-ca8606a4e89e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:58.749236 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6804d39-40e0-41b4-af6a-ca8606a4e89e None admin] PATH_INFO: `/v3/projects/f8a33963b3504bf196b09f575de6e329` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:58.750460 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6804d39-40e0-41b4-af6a-ca8606a4e89e None admin] RBAC: Authorizing `identity:get_project(project_id=f8a33963b3504bf196b09f575de6e329)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:58.752135 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6804d39-40e0-41b4-af6a-ca8606a4e89e None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:58.754182 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 118/247] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 4 09:05:58 2022] GET /identity/v3/projects/f8a33963b3504bf196b09f575de6e329 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:05:58.763421 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9460eb-8199-44a1-adcb-d11d7d26b53e None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:58.763712 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9460eb-8199-44a1-adcb-d11d7d26b53e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:58.765390 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9460eb-8199-44a1-adcb-d11d7d26b53e None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:58.767213 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 119/248] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:05:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:05:58.768342 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9460eb-8199-44a1-adcb-d11d7d26b53e None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:58.770840 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:58.781639 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c16d05-1eb8-4e0a-a605-b4bb62630ff7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:58.850019 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdac11d4-55c6-4dab-ae8c-f77cd793c4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:58.851191 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdac11d4-55c6-4dab-ae8c-f77cd793c4a0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:58.852589 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdac11d4-55c6-4dab-ae8c-f77cd793c4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:58.854035 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdac11d4-55c6-4dab-ae8c-f77cd793c4a0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:58.872068 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdac11d4-55c6-4dab-ae8c-f77cd793c4a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:05:58.881149 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-cdac11d4-55c6-4dab-ae8c-f77cd793c4a0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:05:58.881149 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:58.881149 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:58.881149 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:58.881149 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:58.881149 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:58.881149 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:58.881149 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:58.881149 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:58.881149 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:58.881149 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:58.881149 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:58.881149 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:05:58.881149 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:05:58.881149 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:05:58.881149 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:05:58.882030 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:05:58.882030 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:05:58.882030 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:05:58.882030 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:05:58.882030 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:58.882030 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:58.882030 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:58.882030 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:58.882030 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:58.882030 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:58.882030 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:58.882030 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:58.882030 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:58.882030 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:58.882030 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:58.882030 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:58.882868 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:58.882868 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:58.882868 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:58.882868 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:58.882868 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:05:58.882868 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:05:58.882868 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:05:58.882868 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:05:58.882868 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:05:58.882868 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:05:58.882868 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:05:58.882868 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:05:58.886331 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 130/249] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 4 09:05:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:05:58.887946 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300db6b-e707-4cae-8e36-3171a1e4b62b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:58.888456 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300db6b-e707-4cae-8e36-3171a1e4b62b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:58.893552 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300db6b-e707-4cae-8e36-3171a1e4b62b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:58.928161 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7300db6b-e707-4cae-8e36-3171a1e4b62b None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:05:58.930348 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7300db6b-e707-4cae-8e36-3171a1e4b62b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:05:58.945398 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:58.968954 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9460eb-8199-44a1-adcb-d11d7d26b53e None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:59.016473 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2614be-f4c1-436e-8eb0-d31c88600db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:59.017084 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2614be-f4c1-436e-8eb0-d31c88600db4 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:59.017377 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2614be-f4c1-436e-8eb0-d31c88600db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:59.017658 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2614be-f4c1-436e-8eb0-d31c88600db4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:59.018507 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2614be-f4c1-436e-8eb0-d31c88600db4 None admin] RBAC: Adding query filter params (role.id=56205a5df6514298a69fde673550f0aa, scope.project.id=f8a33963b3504bf196b09f575de6e329, user.id=58287e2295c04e5f89626f1fc39bfb5d) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:59.018824 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2614be-f4c1-436e-8eb0-d31c88600db4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:59.020306 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2614be-f4c1-436e-8eb0-d31c88600db4 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:59.036874 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 120/250] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 4 09:05:58 2022] GET /identity/v3/role_assignments?user.id=58287e2295c04e5f89626f1fc39bfb5d&scope.project.id=f8a33963b3504bf196b09f575de6e329&role.id=56205a5df6514298a69fde673550f0aa => generated 259 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:59.039712 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2614be-f4c1-436e-8eb0-d31c88600db4 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:59.101150 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 131/251] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:05:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:05:59.154822 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7300db6b-e707-4cae-8e36-3171a1e4b62b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:59.237103 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2614be-f4c1-436e-8eb0-d31c88600db4 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:59.289754 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be93498-d6f6-4c32-bb01-62c2272f138d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:59.290967 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be93498-d6f6-4c32-bb01-62c2272f138d None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:59.291532 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be93498-d6f6-4c32-bb01-62c2272f138d None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:59.292013 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be93498-d6f6-4c32-bb01-62c2272f138d None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:59.292970 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be93498-d6f6-4c32-bb01-62c2272f138d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:59.293500 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be93498-d6f6-4c32-bb01-62c2272f138d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:59.295381 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be93498-d6f6-4c32-bb01-62c2272f138d None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:59.305631 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 121/252] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:05:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:59.308140 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:59.319513 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be93498-d6f6-4c32-bb01-62c2272f138d None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:59.335734 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:59.341808 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7300db6b-e707-4cae-8e36-3171a1e4b62b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:59.408803 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4497ab-6bbc-422f-ac83-ca5090b6fa76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:59.409781 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4497ab-6bbc-422f-ac83-ca5090b6fa76 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:59.410072 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4497ab-6bbc-422f-ac83-ca5090b6fa76 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:59.410352 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4497ab-6bbc-422f-ac83-ca5090b6fa76 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:59.430531 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4497ab-6bbc-422f-ac83-ca5090b6fa76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:05:59.443849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-4f4497ab-6bbc-422f-ac83-ca5090b6fa76 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:05:59.443849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:59.443849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:59.443849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:59.443849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:59.443849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:59.443849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:59.443849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:59.443849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:59.443849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:59.443849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:59.443849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:59.443849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:05:59.443849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:05:59.443849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:05:59.443849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:05:59.444763 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:05:59.444763 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:05:59.444763 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:05:59.444763 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:05:59.444763 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:59.444763 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:59.444763 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:59.444763 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:59.444763 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:59.444763 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:59.444763 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:59.444763 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:59.444763 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:59.444763 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:59.444763 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:59.444763 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:59.445660 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:59.445660 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:59.445660 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:59.445660 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:59.445660 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:05:59.445660 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:05:59.445660 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:05:59.445660 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:05:59.445660 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:05:59.445660 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:05:59.445660 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:05:59.445660 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:05:59.448862 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 132/253] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:05:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:05:59.454318 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4497ab-6bbc-422f-ac83-ca5090b6fa76 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:59.493616 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be93498-d6f6-4c32-bb01-62c2272f138d None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:59.552030 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c1927-8c43-4168-98fc-d72acdf5e3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:59.554379 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c1927-8c43-4168-98fc-d72acdf5e3e8 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:59.555153 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c1927-8c43-4168-98fc-d72acdf5e3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:59.556565 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c1927-8c43-4168-98fc-d72acdf5e3e8 None admin] PATH_INFO: `/v3/users/1837335e130248a39830dc4da680f52c` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:59.559496 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7c1927-8c43-4168-98fc-d72acdf5e3e8 None admin] RBAC: Authorizing `identity:get_user(user_id=1837335e130248a39830dc4da680f52c)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:59.562292 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7c1927-8c43-4168-98fc-d72acdf5e3e8 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:59.568473 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 122/254] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 4 09:05:59 2022] GET /identity/v3/users/1837335e130248a39830dc4da680f52c => generated 283 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:59.583922 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c1927-8c43-4168-98fc-d72acdf5e3e8 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:59.649643 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4497ab-6bbc-422f-ac83-ca5090b6fa76 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:59.691177 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4027cda-297c-4ff7-87fa-269464aa7cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:59.692242 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4027cda-297c-4ff7-87fa-269464aa7cdb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:59.692741 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4027cda-297c-4ff7-87fa-269464aa7cdb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:59.693263 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4027cda-297c-4ff7-87fa-269464aa7cdb None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:59.694233 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4027cda-297c-4ff7-87fa-269464aa7cdb None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:59.695770 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4027cda-297c-4ff7-87fa-269464aa7cdb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:59.697947 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4027cda-297c-4ff7-87fa-269464aa7cdb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:59.713632 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 133/255] 10.222.0.71 () {56 vars in 1101 bytes} [Mon Jul 4 09:05:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:05:59.714884 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c9e48-008f-49c7-81da-5471f89c3bc0 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:59.715175 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c9e48-008f-49c7-81da-5471f89c3bc0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:59.715454 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c9e48-008f-49c7-81da-5471f89c3bc0 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:59.716965 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 134/256] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:05:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:05:59.722303 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165cea3e-7db5-4d31-b668-869122342106 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:59.722521 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165cea3e-7db5-4d31-b668-869122342106 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:59.722736 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165cea3e-7db5-4d31-b668-869122342106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:59.727798 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:59.755364 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-165cea3e-7db5-4d31-b668-869122342106 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:05:59.757712 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-165cea3e-7db5-4d31-b668-869122342106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:05:59.766639 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c1927-8c43-4168-98fc-d72acdf5e3e8 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:59.768533 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:59.841231 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa2ed43-5c1b-431b-8dd4-2ea2249feed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:59.841831 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa2ed43-5c1b-431b-8dd4-2ea2249feed9 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:59.842106 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa2ed43-5c1b-431b-8dd4-2ea2249feed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:59.842393 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa2ed43-5c1b-431b-8dd4-2ea2249feed9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:59.854769 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 135/257] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:05:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:05:59.855974 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165cea3e-7db5-4d31-b668-869122342106 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:59.965693 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165cea3e-7db5-4d31-b668-869122342106 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:00.012261 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ed9531-40a9-44be-a75e-dba83f3b9654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:00.014624 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed9531-40a9-44be-a75e-dba83f3b9654 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:00.015824 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed9531-40a9-44be-a75e-dba83f3b9654 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:00.016316 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed9531-40a9-44be-a75e-dba83f3b9654 None admin] PATH_INFO: `/v3/users/abd9f7d1a38340afa59c58abfabbf5f5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:00.018649 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ed9531-40a9-44be-a75e-dba83f3b9654 None admin] RBAC: Authorizing `identity:get_user(user_id=abd9f7d1a38340afa59c58abfabbf5f5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:00.021385 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ed9531-40a9-44be-a75e-dba83f3b9654 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:00.025259 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 136/258] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 4 09:05:59 2022] GET /identity/v3/users/abd9f7d1a38340afa59c58abfabbf5f5 => generated 297 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:00.027481 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ed9531-40a9-44be-a75e-dba83f3b9654 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:00.044031 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa2ed43-5c1b-431b-8dd4-2ea2249feed9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:00.045880 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa2ed43-5c1b-431b-8dd4-2ea2249feed9 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:00.050889 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:00.153345 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ed9531-40a9-44be-a75e-dba83f3b9654 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:00.191319 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5a682a-30cd-49af-886b-d433537f1481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:00.192490 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a682a-30cd-49af-886b-d433537f1481 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:00.192979 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a682a-30cd-49af-886b-d433537f1481 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:00.193488 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a682a-30cd-49af-886b-d433537f1481 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:00.205334 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5a682a-30cd-49af-886b-d433537f1481 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:06:00.215043 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-1c5a682a-30cd-49af-886b-d433537f1481 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:06:00.215043 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:00.215043 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:00.215043 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:00.215043 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:00.215043 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:00.215043 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:00.215043 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:00.215043 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:00.215043 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:00.215043 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:00.215043 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:00.215043 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:06:00.215043 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:06:00.215043 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:06:00.215043 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:06:00.215953 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:06:00.215953 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:06:00.215953 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:06:00.215953 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:06:00.215953 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:00.215953 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:00.215953 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:00.215953 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:00.215953 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:00.215953 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:00.215953 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:00.215953 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:00.215953 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:00.215953 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:00.215953 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:00.215953 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:00.216755 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:00.216755 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:00.216755 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:00.216755 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:00.216755 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:06:00.216755 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:06:00.216755 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:06:00.216755 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:06:00.216755 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:06:00.216755 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:06:00.216755 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:06:00.216755 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:00.219359 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 137/259] 10.222.0.71 () {56 vars in 1114 bytes} [Mon Jul 4 09:06:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:00.220455 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5a682a-30cd-49af-886b-d433537f1481 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:00.286204 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:00.325506 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:00.335407 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5a682a-30cd-49af-886b-d433537f1481 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:00.374796 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b846964d-ba31-4d9e-a784-428798d517f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:00.376032 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b846964d-ba31-4d9e-a784-428798d517f2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:00.376546 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b846964d-ba31-4d9e-a784-428798d517f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:00.377057 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b846964d-ba31-4d9e-a784-428798d517f2 None admin] PATH_INFO: `/v3/projects/f8a33963b3504bf196b09f575de6e329` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:00.377497 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 123/260] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:05:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 794 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 04 09:06:00.378356 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa2ed43-5c1b-431b-8dd4-2ea2249feed9 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:00.379015 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b846964d-ba31-4d9e-a784-428798d517f2 None admin] RBAC: Authorizing `identity:get_project(project_id=f8a33963b3504bf196b09f575de6e329)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:00.380665 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b846964d-ba31-4d9e-a784-428798d517f2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:00.382682 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 138/261] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 4 09:06:00 2022] GET /identity/v3/projects/f8a33963b3504bf196b09f575de6e329 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:00.383666 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b846964d-ba31-4d9e-a784-428798d517f2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:00.525730 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa2ed43-5c1b-431b-8dd4-2ea2249feed9 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:00.551270 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b846964d-ba31-4d9e-a784-428798d517f2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:00.560773 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bdfac2-e088-4ce8-8e7d-54897739e93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:00.561485 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bdfac2-e088-4ce8-8e7d-54897739e93d None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:00.561773 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bdfac2-e088-4ce8-8e7d-54897739e93d None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:00.562053 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bdfac2-e088-4ce8-8e7d-54897739e93d None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:00.562811 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bdfac2-e088-4ce8-8e7d-54897739e93d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:00.563117 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bdfac2-e088-4ce8-8e7d-54897739e93d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:00.564524 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bdfac2-e088-4ce8-8e7d-54897739e93d None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:00.573139 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 124/262] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 4 09:06:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:00.574050 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bdfac2-e088-4ce8-8e7d-54897739e93d None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:00.611411 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:00.646559 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dce5d7b-358f-48b5-9832-b82495b88959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:00.647508 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce5d7b-358f-48b5-9832-b82495b88959 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:00.647799 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce5d7b-358f-48b5-9832-b82495b88959 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:00.648083 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce5d7b-358f-48b5-9832-b82495b88959 None admin] PATH_INFO: `/v3/projects/f8a33963b3504bf196b09f575de6e329` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:00.653809 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dce5d7b-358f-48b5-9832-b82495b88959 None admin] RBAC: Authorizing `identity:get_project(project_id=f8a33963b3504bf196b09f575de6e329)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:00.655281 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dce5d7b-358f-48b5-9832-b82495b88959 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:00.657433 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 139/263] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 4 09:06:00 2022] GET /identity/v3/projects/f8a33963b3504bf196b09f575de6e329 => generated 302 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:00.658239 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dce5d7b-358f-48b5-9832-b82495b88959 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:00.667408 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:00.687449 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bdfac2-e088-4ce8-8e7d-54897739e93d None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:00.722937 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1352fbb-182f-4ee2-a542-06e1db51903b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:00.724314 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1352fbb-182f-4ee2-a542-06e1db51903b None admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:00.724609 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1352fbb-182f-4ee2-a542-06e1db51903b None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:00.724900 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1352fbb-182f-4ee2-a542-06e1db51903b None admin] PATH_INFO: `/v3/projects/f8a33963b3504bf196b09f575de6e329/users/abd9f7d1a38340afa59c58abfabbf5f5/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:00.727111 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1352fbb-182f-4ee2-a542-06e1db51903b None admin] RBAC: Authorizing `identity:create_grant(project_id=f8a33963b3504bf196b09f575de6e329, user_id=abd9f7d1a38340afa59c58abfabbf5f5, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:00.729369 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1352fbb-182f-4ee2-a542-06e1db51903b None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:00.751591 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 125/264] 10.222.0.71 () {58 vars in 1439 bytes} [Mon Jul 4 09:06:00 2022] PUT /identity/v3/projects/f8a33963b3504bf196b09f575de6e329/users/abd9f7d1a38340afa59c58abfabbf5f5/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:06:00.754203 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1352fbb-182f-4ee2-a542-06e1db51903b None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:00.912268 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dce5d7b-358f-48b5-9832-b82495b88959 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:00.973780 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:00.980663 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1352fbb-182f-4ee2-a542-06e1db51903b None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:00.991198 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8b650e-4a3e-4fb1-a811-864c513af8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:00.992123 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b650e-4a3e-4fb1-a811-864c513af8b4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:00.993849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b650e-4a3e-4fb1-a811-864c513af8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:00.993849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b650e-4a3e-4fb1-a811-864c513af8b4 None admin] PATH_INFO: `/v3/users/1837335e130248a39830dc4da680f52c` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:00.994863 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8b650e-4a3e-4fb1-a811-864c513af8b4 None admin] RBAC: Authorizing `identity:get_user(user_id=1837335e130248a39830dc4da680f52c)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:00.997275 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8b650e-4a3e-4fb1-a811-864c513af8b4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:00.999327 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 140/265] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 4 09:06:00 2022] GET /identity/v3/users/1837335e130248a39830dc4da680f52c => generated 283 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:01.005081 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8b650e-4a3e-4fb1-a811-864c513af8b4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:01.023052 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ab89d2-0aed-43d4-8041-1771330fe70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:01.024190 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab89d2-0aed-43d4-8041-1771330fe70e None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:01.024668 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab89d2-0aed-43d4-8041-1771330fe70e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:01.025904 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab89d2-0aed-43d4-8041-1771330fe70e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:01.027076 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ab89d2-0aed-43d4-8041-1771330fe70e None admin] RBAC: Adding query filter params (role.id=f36e053b442540f08041ac7878760f65, scope.project.id=f8a33963b3504bf196b09f575de6e329, user.id=1837335e130248a39830dc4da680f52c) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:01.027403 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ab89d2-0aed-43d4-8041-1771330fe70e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:01.029355 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ab89d2-0aed-43d4-8041-1771330fe70e None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:01.037908 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 126/266] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 4 09:06:00 2022] GET /identity/v3/role_assignments?user.id=1837335e130248a39830dc4da680f52c&scope.project.id=f8a33963b3504bf196b09f575de6e329&role.id=f36e053b442540f08041ac7878760f65 => generated 608 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:01.044248 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a1490c-51c8-4983-bd9e-c60e99d0bb20 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:01.044545 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a1490c-51c8-4983-bd9e-c60e99d0bb20 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:01.044835 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a1490c-51c8-4983-bd9e-c60e99d0bb20 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:01.046348 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 127/267] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:06:01.052561 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c59ec-0e0b-4c2b-a0ea-e83d5a6ab811 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:01.052853 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c59ec-0e0b-4c2b-a0ea-e83d5a6ab811 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:01.053150 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c59ec-0e0b-4c2b-a0ea-e83d5a6ab811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:01.076007 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-f40c59ec-0e0b-4c2b-a0ea-e83d5a6ab811 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:01.077288 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f40c59ec-0e0b-4c2b-a0ea-e83d5a6ab811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:01.099201 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:01.139017 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8b650e-4a3e-4fb1-a811-864c513af8b4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:01.157639 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 128/268] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:06:01.159849 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ca5336-23aa-46cf-ab9b-bcc22bad9744 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:01.160141 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ca5336-23aa-46cf-ab9b-bcc22bad9744 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:01.160424 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ca5336-23aa-46cf-ab9b-bcc22bad9744 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:01.161744 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 129/269] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:01.167831 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7f095-38d6-45e7-ae0c-841192f11246 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:01.168173 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7f095-38d6-45e7-ae0c-841192f11246 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:01.168556 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7f095-38d6-45e7-ae0c-841192f11246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:01.189662 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-ddf7f095-38d6-45e7-ae0c-841192f11246 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:01.191085 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ddf7f095-38d6-45e7-ae0c-841192f11246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:01.192710 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7a9fda-d5c3-466f-bf13-156d01a93745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:01.193582 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a9fda-d5c3-466f-bf13-156d01a93745 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:01.193949 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a9fda-d5c3-466f-bf13-156d01a93745 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:01.194308 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a9fda-d5c3-466f-bf13-156d01a93745 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:01.268464 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7a9fda-d5c3-466f-bf13-156d01a93745 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:01.270195 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7a9fda-d5c3-466f-bf13-156d01a93745 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:01.283694 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 130/270] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:06:01.284568 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf7f095-38d6-45e7-ae0c-841192f11246 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:01.334019 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:01.365434 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:01.406190 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 141/271] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:06:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:06:01.407148 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7a9fda-d5c3-466f-bf13-156d01a93745 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:01.430223 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf7f095-38d6-45e7-ae0c-841192f11246 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:01.475433 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156a3166-ec06-44e3-8be6-597ffe5f903a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:01.476641 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156a3166-ec06-44e3-8be6-597ffe5f903a None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:01.477119 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156a3166-ec06-44e3-8be6-597ffe5f903a None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:01.477540 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156a3166-ec06-44e3-8be6-597ffe5f903a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:01.489476 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156a3166-ec06-44e3-8be6-597ffe5f903a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:06:01.501107 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-156a3166-ec06-44e3-8be6-597ffe5f903a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:06:01.501107 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:01.501107 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:01.501107 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:01.501107 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:01.501107 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:01.501107 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:01.501107 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:01.501107 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:01.501107 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:01.501107 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:01.501107 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:01.501107 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:06:01.501107 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:06:01.501107 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:06:01.501107 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:06:01.502219 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:06:01.502219 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:06:01.502219 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:06:01.502219 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:06:01.502219 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:01.502219 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:01.502219 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:01.502219 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:01.502219 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:01.502219 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:01.502219 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:01.502219 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:01.502219 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:01.502219 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:01.502219 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:01.502219 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:01.503215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:01.503215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:01.503215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:01.503215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:01.503215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:06:01.503215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:06:01.503215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:06:01.503215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:06:01.503215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:06:01.503215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:06:01.503215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:06:01.503215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:01.505528 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 131/272] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:06:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:01.506539 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156a3166-ec06-44e3-8be6-597ffe5f903a None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:01.524261 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7a9fda-d5c3-466f-bf13-156d01a93745 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:01.561403 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adf8e88-888c-4fcb-bb11-9dc2d461246d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:01.562208 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf8e88-888c-4fcb-bb11-9dc2d461246d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:01.562498 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf8e88-888c-4fcb-bb11-9dc2d461246d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:01.562781 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf8e88-888c-4fcb-bb11-9dc2d461246d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:01.563972 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adf8e88-888c-4fcb-bb11-9dc2d461246d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:01.565468 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adf8e88-888c-4fcb-bb11-9dc2d461246d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:01.567099 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 142/273] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:06:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:01.567874 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adf8e88-888c-4fcb-bb11-9dc2d461246d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:01.615147 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:01.619276 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:01.645060 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156a3166-ec06-44e3-8be6-597ffe5f903a None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:01.678039 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adf8e88-888c-4fcb-bb11-9dc2d461246d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:01.692553 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43f7159-326c-4c3a-99a8-c36646ca3b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:01.693404 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43f7159-326c-4c3a-99a8-c36646ca3b1f None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:01.693697 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43f7159-326c-4c3a-99a8-c36646ca3b1f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:01.693984 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43f7159-326c-4c3a-99a8-c36646ca3b1f None admin] PATH_INFO: `/v3/projects/f8a33963b3504bf196b09f575de6e329` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:01.695919 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43f7159-326c-4c3a-99a8-c36646ca3b1f None admin] RBAC: Authorizing `identity:get_project(project_id=f8a33963b3504bf196b09f575de6e329)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:01.698270 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43f7159-326c-4c3a-99a8-c36646ca3b1f None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:01.700737 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 132/274] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 4 09:06:01 2022] GET /identity/v3/projects/f8a33963b3504bf196b09f575de6e329 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:01.701531 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43f7159-326c-4c3a-99a8-c36646ca3b1f None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:01.719669 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebdf644-1c64-4388-8403-0a415bccc7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:01.720710 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebdf644-1c64-4388-8403-0a415bccc7b8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:01.721202 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebdf644-1c64-4388-8403-0a415bccc7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:01.721825 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebdf644-1c64-4388-8403-0a415bccc7b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:01.722822 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebdf644-1c64-4388-8403-0a415bccc7b8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:01.723304 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebdf644-1c64-4388-8403-0a415bccc7b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:01.724862 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebdf644-1c64-4388-8403-0a415bccc7b8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:01.734094 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 143/275] 10.222.0.71 () {56 vars in 1101 bytes} [Mon Jul 4 09:06:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:01.735284 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebdf644-1c64-4388-8403-0a415bccc7b8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:01.829991 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43f7159-326c-4c3a-99a8-c36646ca3b1f None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:01.844696 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebdf644-1c64-4388-8403-0a415bccc7b8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:01.846263 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:01.868221 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7780fef7-0ad5-4652-9690-75d88ffcb685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:01.869028 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7780fef7-0ad5-4652-9690-75d88ffcb685 None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:01.869354 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7780fef7-0ad5-4652-9690-75d88ffcb685 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:01.869636 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7780fef7-0ad5-4652-9690-75d88ffcb685 None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:01.870361 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7780fef7-0ad5-4652-9690-75d88ffcb685 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:01.871828 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7780fef7-0ad5-4652-9690-75d88ffcb685 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:01.883445 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf5b66f-9caa-4144-863f-d205cb64b558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:01.885266 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5b66f-9caa-4144-863f-d205cb64b558 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:01.885612 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5b66f-9caa-4144-863f-d205cb64b558 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:01.885941 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5b66f-9caa-4144-863f-d205cb64b558 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:01.887032 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf5b66f-9caa-4144-863f-d205cb64b558 None admin] RBAC: Adding query filter params (role.id=5fb58444d5604bf689014e27dccacb45, scope.project.id=f8a33963b3504bf196b09f575de6e329, user.id=1837335e130248a39830dc4da680f52c) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:01.887473 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf5b66f-9caa-4144-863f-d205cb64b558 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:01.889157 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf5b66f-9caa-4144-863f-d205cb64b558 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:01.895823 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 144/276] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 4 09:06:01 2022] GET /identity/v3/role_assignments?user.id=1837335e130248a39830dc4da680f52c&scope.project.id=f8a33963b3504bf196b09f575de6e329&role.id=5fb58444d5604bf689014e27dccacb45 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:01.896461 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf5b66f-9caa-4144-863f-d205cb64b558 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:01.897602 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 133/277] 10.222.0.71 () {60 vars in 1132 bytes} [Mon Jul 4 09:06:01 2022] POST /identity/v3/users => generated 293 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:06:01.899084 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b28f94-283e-4a50-aa68-af8e295c2ecd None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:01.899544 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b28f94-283e-4a50-aa68-af8e295c2ecd None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:01.900539 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b28f94-283e-4a50-aa68-af8e295c2ecd None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:01.902359 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 134/278] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:01.908407 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c50a7b-c471-4a64-b567-fc2b3ad2b599 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:01.909339 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c50a7b-c471-4a64-b567-fc2b3ad2b599 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:01.909819 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c50a7b-c471-4a64-b567-fc2b3ad2b599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:01.919643 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:01.939681 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-62c50a7b-c471-4a64-b567-fc2b3ad2b599 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:01.941623 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-62c50a7b-c471-4a64-b567-fc2b3ad2b599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:02.099703 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 135/279] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:06:02.123364 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf5b66f-9caa-4144-863f-d205cb64b558 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:02.125456 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c50a7b-c471-4a64-b567-fc2b3ad2b599 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:02.175998 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b5872c-634e-41b1-a4d0-130509ce73c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:02.176937 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5872c-634e-41b1-a4d0-130509ce73c0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:02.177259 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5872c-634e-41b1-a4d0-130509ce73c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:02.177543 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5872c-634e-41b1-a4d0-130509ce73c0 None admin] PATH_INFO: `/v3/users/c410015dcf824f3397fbd9d0606bfa2c` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:02.204864 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b5872c-634e-41b1-a4d0-130509ce73c0 None admin] RBAC: Authorizing `identity:get_user(user_id=c410015dcf824f3397fbd9d0606bfa2c)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:02.206938 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b5872c-634e-41b1-a4d0-130509ce73c0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:02.210361 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 145/280] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 4 09:06:01 2022] GET /identity/v3/users/c410015dcf824f3397fbd9d0606bfa2c => generated 293 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:02.217785 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b5872c-634e-41b1-a4d0-130509ce73c0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:02.250763 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:02.267518 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c50a7b-c471-4a64-b567-fc2b3ad2b599 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:02.317210 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce7d8a7-211d-4577-9296-44414e9a3ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:02.318350 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7d8a7-211d-4577-9296-44414e9a3ef0 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:02.318801 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7d8a7-211d-4577-9296-44414e9a3ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:02.319242 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7d8a7-211d-4577-9296-44414e9a3ef0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:02.322676 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:02.332961 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce7d8a7-211d-4577-9296-44414e9a3ef0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:06:02.342028 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b5872c-634e-41b1-a4d0-130509ce73c0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:02.343880 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-cce7d8a7-211d-4577-9296-44414e9a3ef0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 09:06:02.343880 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:02.343880 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:02.343880 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:02.343880 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:02.343880 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:02.343880 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:02.343880 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:02.343880 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:02.343880 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:02.343880 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:02.343880 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:02.343880 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:06:02.343880 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:06:02.343880 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:06:02.343880 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:06:02.344757 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:06:02.344757 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:06:02.344757 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:06:02.344757 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:06:02.344757 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:02.344757 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:02.344757 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:02.344757 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:02.344757 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:02.344757 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:02.344757 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:02.344757 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:02.344757 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:02.344757 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:02.344757 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:02.344757 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:02.345598 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:02.345598 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:02.345598 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:02.345598 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:02.345598 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:06:02.345598 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:06:02.345598 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:06:02.345598 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:06:02.345598 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:06:02.345598 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:06:02.345598 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 09:06:02.345598 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:02.348059 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 136/281] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:06:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:06:02.356163 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce7d8a7-211d-4577-9296-44414e9a3ef0 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:02.408154 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65d94a4-a476-4344-b476-e40362b40b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:02.409878 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d94a4-a476-4344-b476-e40362b40b5d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:02.410366 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d94a4-a476-4344-b476-e40362b40b5d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:02.410839 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d94a4-a476-4344-b476-e40362b40b5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:02.411932 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65d94a4-a476-4344-b476-e40362b40b5d None admin] RBAC: Adding query filter params (role.id=8692e99e9463459f9b7caa96b3051b93, user.id=c410015dcf824f3397fbd9d0606bfa2c, scope.system=all) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:02.412874 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65d94a4-a476-4344-b476-e40362b40b5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:02.415069 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65d94a4-a476-4344-b476-e40362b40b5d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:02.422041 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 146/282] 10.222.0.71 () {56 vars in 1308 bytes} [Mon Jul 4 09:06:02 2022] GET /identity/v3/role_assignments?user.id=c410015dcf824f3397fbd9d0606bfa2c&scope.system=all&role.id=8692e99e9463459f9b7caa96b3051b93 => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:06:02.531096 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce7d8a7-211d-4577-9296-44414e9a3ef0 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:02.577812 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311ea369-58d6-4925-90e0-c115e95d5285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:02.578869 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ea369-58d6-4925-90e0-c115e95d5285 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:02.579356 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ea369-58d6-4925-90e0-c115e95d5285 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:02.579835 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ea369-58d6-4925-90e0-c115e95d5285 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:02.580806 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311ea369-58d6-4925-90e0-c115e95d5285 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:02.581374 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311ea369-58d6-4925-90e0-c115e95d5285 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:02.583113 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311ea369-58d6-4925-90e0-c115e95d5285 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:02.599386 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 137/283] 10.222.0.71 () {56 vars in 1101 bytes} [Mon Jul 4 09:06:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:02.605370 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65d94a4-a476-4344-b476-e40362b40b5d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:02.722742 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:02.740833 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65d94a4-a476-4344-b476-e40362b40b5d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:02.784562 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02cf62c-b68d-42ee-9edc-1465bd7f4b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:02.785481 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02cf62c-b68d-42ee-9edc-1465bd7f4b5e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:02.785889 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02cf62c-b68d-42ee-9edc-1465bd7f4b5e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:02.786166 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02cf62c-b68d-42ee-9edc-1465bd7f4b5e None admin] PATH_INFO: `/v3/users/58287e2295c04e5f89626f1fc39bfb5d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:02.787387 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02cf62c-b68d-42ee-9edc-1465bd7f4b5e None admin] RBAC: Authorizing `identity:get_user(user_id=58287e2295c04e5f89626f1fc39bfb5d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:02.788813 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02cf62c-b68d-42ee-9edc-1465bd7f4b5e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:02.791315 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 147/284] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 4 09:06:02 2022] GET /identity/v3/users/58287e2295c04e5f89626f1fc39bfb5d => generated 297 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:02.796764 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311ea369-58d6-4925-90e0-c115e95d5285 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:02.865736 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:02.919818 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311ea369-58d6-4925-90e0-c115e95d5285 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:02.964904 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a591cb64-13b5-46c8-a4d0-1260cf25a875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:02.965815 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a591cb64-13b5-46c8-a4d0-1260cf25a875 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:02.966099 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a591cb64-13b5-46c8-a4d0-1260cf25a875 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:02.966379 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a591cb64-13b5-46c8-a4d0-1260cf25a875 None admin] PATH_INFO: `/v3/projects/f8a33963b3504bf196b09f575de6e329` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:02.967599 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a591cb64-13b5-46c8-a4d0-1260cf25a875 None admin] RBAC: Authorizing `identity:get_project(project_id=f8a33963b3504bf196b09f575de6e329)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:02.969083 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a591cb64-13b5-46c8-a4d0-1260cf25a875 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:02.971108 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 138/285] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 4 09:06:02 2022] GET /identity/v3/projects/f8a33963b3504bf196b09f575de6e329 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:02.976626 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02cf62c-b68d-42ee-9edc-1465bd7f4b5e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:03.127670 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02cf62c-b68d-42ee-9edc-1465bd7f4b5e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:03.176378 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fa3319-cc28-492f-a324-e339a62dafa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:03.177292 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa3319-cc28-492f-a324-e339a62dafa4 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:03.177575 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa3319-cc28-492f-a324-e339a62dafa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:03.177849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa3319-cc28-492f-a324-e339a62dafa4 None admin] PATH_INFO: `/v3/projects/f8a33963b3504bf196b09f575de6e329/users/58287e2295c04e5f89626f1fc39bfb5d/roles/56205a5df6514298a69fde673550f0aa` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:03.183333 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fa3319-cc28-492f-a324-e339a62dafa4 None admin] RBAC: Authorizing `identity:create_grant(project_id=f8a33963b3504bf196b09f575de6e329, user_id=58287e2295c04e5f89626f1fc39bfb5d, role_id=56205a5df6514298a69fde673550f0aa)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:03.184763 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fa3319-cc28-492f-a324-e339a62dafa4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:03.209290 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 148/286] 10.222.0.71 () {58 vars in 1439 bytes} [Mon Jul 4 09:06:02 2022] PUT /identity/v3/projects/f8a33963b3504bf196b09f575de6e329/users/58287e2295c04e5f89626f1fc39bfb5d/roles/56205a5df6514298a69fde673550f0aa => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:06:03.380180 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a08bc90-f487-44e5-afbe-df0a7e9dbf8c None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:03.380525 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a08bc90-f487-44e5-afbe-df0a7e9dbf8c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:03.382393 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a08bc90-f487-44e5-afbe-df0a7e9dbf8c None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:03.382393 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 139/287] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:03.391425 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261fd458-165e-48ba-a5dd-36e96783023e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:03.391425 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261fd458-165e-48ba-a5dd-36e96783023e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:03.391425 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261fd458-165e-48ba-a5dd-36e96783023e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:03.414251 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-261fd458-165e-48ba-a5dd-36e96783023e None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:03.416443 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-261fd458-165e-48ba-a5dd-36e96783023e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:03.434412 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:03.499938 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 149/288] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:06:03.522705 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a08bc90-f487-44e5-afbe-df0a7e9dbf8c None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:03.677646 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a08bc90-f487-44e5-afbe-df0a7e9dbf8c None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:03.708983 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:03.723215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb4ceee-7430-4693-bbc6-0a46bdb2aaae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:03.724348 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb4ceee-7430-4693-bbc6-0a46bdb2aaae None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:03.724832 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb4ceee-7430-4693-bbc6-0a46bdb2aaae None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:03.725333 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb4ceee-7430-4693-bbc6-0a46bdb2aaae None admin] PATH_INFO: `/v3/roles/member` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:03.743998 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb4ceee-7430-4693-bbc6-0a46bdb2aaae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:06:03.752033 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-4eb4ceee-7430-4693-bbc6-0a46bdb2aaae None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:06:03.752033 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:03.752033 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:03.752033 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:03.752033 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:03.752033 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:03.752033 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:03.752033 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:03.752033 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:03.752033 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:03.752033 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:03.752033 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:03.752033 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:06:03.752033 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:06:03.752033 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:06:03.752033 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:06:03.752907 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:06:03.752907 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:06:03.752907 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:06:03.752907 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:06:03.752907 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:03.752907 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:03.752907 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:03.752907 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:03.752907 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:03.752907 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:03.752907 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:03.752907 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:03.752907 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:03.752907 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:03.752907 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:03.752907 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:03.753766 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:03.753766 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:03.753766 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:03.753766 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:03.753766 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:06:03.753766 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:06:03.753766 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:06:03.753766 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:06:03.753766 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:06:03.753766 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:06:03.753766 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:06:03.753766 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:03.756097 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 140/289] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:06:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:03.762057 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261fd458-165e-48ba-a5dd-36e96783023e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:03.938672 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261fd458-165e-48ba-a5dd-36e96783023e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:03.995669 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3926d552-ecde-400e-8dd4-0cf9ac7a2293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:03.996355 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3926d552-ecde-400e-8dd4-0cf9ac7a2293 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:03.996647 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3926d552-ecde-400e-8dd4-0cf9ac7a2293 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:03.996928 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3926d552-ecde-400e-8dd4-0cf9ac7a2293 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:03.997718 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3926d552-ecde-400e-8dd4-0cf9ac7a2293 None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:03.998030 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3926d552-ecde-400e-8dd4-0cf9ac7a2293 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:03.999408 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3926d552-ecde-400e-8dd4-0cf9ac7a2293 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:04.013269 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 150/290] 10.222.0.71 () {56 vars in 1101 bytes} [Mon Jul 4 09:06:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:04.018464 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb4ceee-7430-4693-bbc6-0a46bdb2aaae None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:04.178891 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb4ceee-7430-4693-bbc6-0a46bdb2aaae None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:04.220268 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2113275-63df-4281-8200-69fc160f7ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:04.221565 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2113275-63df-4281-8200-69fc160f7ee7 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:04.222054 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2113275-63df-4281-8200-69fc160f7ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:04.222530 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2113275-63df-4281-8200-69fc160f7ee7 None admin] PATH_INFO: `/v3/users/abd9f7d1a38340afa59c58abfabbf5f5` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:04.223934 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2113275-63df-4281-8200-69fc160f7ee7 None admin] RBAC: Authorizing `identity:get_user(user_id=abd9f7d1a38340afa59c58abfabbf5f5)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:04.225689 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2113275-63df-4281-8200-69fc160f7ee7 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:04.227685 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 141/291] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 4 09:06:04 2022] GET /identity/v3/users/abd9f7d1a38340afa59c58abfabbf5f5 => generated 297 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:04.234060 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3926d552-ecde-400e-8dd4-0cf9ac7a2293 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:04.259696 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:04.400856 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3926d552-ecde-400e-8dd4-0cf9ac7a2293 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:04.444598 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf992701-c474-4115-a965-25800d5dcf9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:04.445376 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf992701-c474-4115-a965-25800d5dcf9a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:04.445574 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf992701-c474-4115-a965-25800d5dcf9a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:04.445860 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf992701-c474-4115-a965-25800d5dcf9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:04.540603 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf992701-c474-4115-a965-25800d5dcf9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:04.542599 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf992701-c474-4115-a965-25800d5dcf9a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:04.558764 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:04.688338 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 151/292] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:06:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:06:04.693816 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2113275-63df-4281-8200-69fc160f7ee7 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:04.706405 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c581d0-b5cf-4d15-95d9-15e881d25c4a None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:04.706693 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c581d0-b5cf-4d15-95d9-15e881d25c4a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:04.707140 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c581d0-b5cf-4d15-95d9-15e881d25c4a None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:04.708772 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 152/293] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:06:04.714848 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f1b696-3c16-4dc9-aeb1-8aa36937cc6d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:04.715415 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f1b696-3c16-4dc9-aeb1-8aa36937cc6d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:04.715862 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f1b696-3c16-4dc9-aeb1-8aa36937cc6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:04.741113 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-33f1b696-3c16-4dc9-aeb1-8aa36937cc6d None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:04.742574 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-33f1b696-3c16-4dc9-aeb1-8aa36937cc6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:04.814436 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:04.876200 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 153/294] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:06:04.881232 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda0c41-9962-4600-85cf-408aa65cb002 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:04.881717 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda0c41-9962-4600-85cf-408aa65cb002 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:04.882138 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda0c41-9962-4600-85cf-408aa65cb002 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:04.883821 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 154/295] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:06:04.889544 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeece57f-4d97-4577-aa36-582e477d7317 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:04.890090 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeece57f-4d97-4577-aa36-582e477d7317 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:04.890582 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeece57f-4d97-4577-aa36-582e477d7317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:04.908059 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-aeece57f-4d97-4577-aa36-582e477d7317 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:04.909620 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-aeece57f-4d97-4577-aa36-582e477d7317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:04.910515 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2113275-63df-4281-8200-69fc160f7ee7 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:04.955890 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:04.957471 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f055b76-650d-46c1-a0b7-6023b8526507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:04.958689 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f055b76-650d-46c1-a0b7-6023b8526507 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:04.958980 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f055b76-650d-46c1-a0b7-6023b8526507 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:04.959281 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f055b76-650d-46c1-a0b7-6023b8526507 None admin] PATH_INFO: `/v3/projects/f8a33963b3504bf196b09f575de6e329` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:04.960762 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f055b76-650d-46c1-a0b7-6023b8526507 None admin] RBAC: Authorizing `identity:get_project(project_id=f8a33963b3504bf196b09f575de6e329)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:04.962868 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f055b76-650d-46c1-a0b7-6023b8526507 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:04.969086 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 142/296] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 4 09:06:04 2022] GET /identity/v3/projects/f8a33963b3504bf196b09f575de6e329 => generated 302 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:04.969086 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f055b76-650d-46c1-a0b7-6023b8526507 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:05.013395 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 155/297] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:06:05.014153 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeece57f-4d97-4577-aa36-582e477d7317 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:05.109561 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f055b76-650d-46c1-a0b7-6023b8526507 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:05.125960 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeece57f-4d97-4577-aa36-582e477d7317 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:05.130557 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:05.145952 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb1dd24-9f5c-4e87-ad28-64422f1ebc0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:05.146745 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb1dd24-9f5c-4e87-ad28-64422f1ebc0e None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:05.147030 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb1dd24-9f5c-4e87-ad28-64422f1ebc0e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:05.147307 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb1dd24-9f5c-4e87-ad28-64422f1ebc0e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:05.148554 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb1dd24-9f5c-4e87-ad28-64422f1ebc0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:05.150571 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb1dd24-9f5c-4e87-ad28-64422f1ebc0e None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:05.152471 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 143/298] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:06:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:05.157463 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb1dd24-9f5c-4e87-ad28-64422f1ebc0e None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:05.161529 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542738a3-100b-4c1e-8ff3-d24483b0801a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:05.162191 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542738a3-100b-4c1e-8ff3-d24483b0801a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:05.162474 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542738a3-100b-4c1e-8ff3-d24483b0801a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:05.162749 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542738a3-100b-4c1e-8ff3-d24483b0801a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:05.163578 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542738a3-100b-4c1e-8ff3-d24483b0801a None admin] RBAC: Adding query filter params (role.id=8692e99e9463459f9b7caa96b3051b93, scope.project.id=f8a33963b3504bf196b09f575de6e329, user.id=abd9f7d1a38340afa59c58abfabbf5f5) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:05.163889 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542738a3-100b-4c1e-8ff3-d24483b0801a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:05.165362 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542738a3-100b-4c1e-8ff3-d24483b0801a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:05.172494 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 156/299] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 4 09:06:05 2022] GET /identity/v3/role_assignments?user.id=abd9f7d1a38340afa59c58abfabbf5f5&scope.project.id=f8a33963b3504bf196b09f575de6e329&role.id=8692e99e9463459f9b7caa96b3051b93 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:05.173594 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf9805-2019-4460-9842-dd17d4e5b6eb None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:05.173879 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf9805-2019-4460-9842-dd17d4e5b6eb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:05.174155 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf9805-2019-4460-9842-dd17d4e5b6eb None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:05.179693 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 157/300] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:05.180664 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadf9805-2019-4460-9842-dd17d4e5b6eb None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:05.221808 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:05.310848 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb1dd24-9f5c-4e87-ad28-64422f1ebc0e None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:05.323736 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadf9805-2019-4460-9842-dd17d4e5b6eb None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:05.356611 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff6469-d4e6-4e5d-a1ee-111c73172bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:05.357812 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff6469-d4e6-4e5d-a1ee-111c73172bfd None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:05.358304 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff6469-d4e6-4e5d-a1ee-111c73172bfd None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:05.358785 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff6469-d4e6-4e5d-a1ee-111c73172bfd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:05.367731 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a034ac-c667-4d26-9274-5d745c687cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:05.368542 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a034ac-c667-4d26-9274-5d745c687cd1 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:05.369389 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a034ac-c667-4d26-9274-5d745c687cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:05.369827 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a034ac-c667-4d26-9274-5d745c687cd1 None admin] PATH_INFO: `/v3/groups` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:05.370541 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a034ac-c667-4d26-9274-5d745c687cd1 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:05.371974 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a034ac-c667-4d26-9274-5d745c687cd1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:05.372079 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fff6469-d4e6-4e5d-a1ee-111c73172bfd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:06:05.381389 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-8fff6469-d4e6-4e5d-a1ee-111c73172bfd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:06:05.381389 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:05.381389 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:05.381389 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:05.381389 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:05.381389 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:05.381389 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:05.381389 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:05.381389 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:05.381389 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:05.381389 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:05.381389 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:05.381389 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:06:05.381389 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:06:05.381389 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:06:05.381389 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:06:05.382379 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:06:05.382379 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:06:05.382379 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:06:05.382379 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:06:05.382379 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:05.382379 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:05.382379 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:05.382379 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:05.382379 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:05.382379 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:05.382379 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:05.382379 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:05.382379 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:05.382379 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:05.382379 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:05.382379 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:05.383170 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:05.383170 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:05.383170 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:05.383170 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:05.383170 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:06:05.383170 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:06:05.383170 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:06:05.383170 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:06:05.383170 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:06:05.383170 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:06:05.383170 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:06:05.383170 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:05.383778 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 144/301] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:06:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:05.385267 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ba6a29-cbae-425b-ab3d-a216568b76cc None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:05.385267 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ba6a29-cbae-425b-ab3d-a216568b76cc None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:05.385449 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ba6a29-cbae-425b-ab3d-a216568b76cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:05.403338 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 158/302] 10.222.0.71 () {60 vars in 1134 bytes} [Mon Jul 4 09:06:05 2022] POST /identity/v3/groups => generated 229 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:06:05.404211 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a034ac-c667-4d26-9274-5d745c687cd1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:05.409594 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-90ba6a29-cbae-425b-ab3d-a216568b76cc None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:05.410831 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-90ba6a29-cbae-425b-ab3d-a216568b76cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:05.474569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:05.513101 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 145/303] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:06:05.537899 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ba6a29-cbae-425b-ab3d-a216568b76cc None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:05.552977 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:05.567917 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a034ac-c667-4d26-9274-5d745c687cd1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:05.610193 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d7e999-6291-4191-b75d-e118e6108088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:05.610910 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d7e999-6291-4191-b75d-e118e6108088 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:05.611377 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d7e999-6291-4191-b75d-e118e6108088 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:05.611856 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d7e999-6291-4191-b75d-e118e6108088 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:05.613923 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d7e999-6291-4191-b75d-e118e6108088 None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:05.613923 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d7e999-6291-4191-b75d-e118e6108088 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:05.616395 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d7e999-6291-4191-b75d-e118e6108088 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:05.630161 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 159/304] 10.222.0.71 () {56 vars in 1101 bytes} [Mon Jul 4 09:06:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:05.635518 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d7e999-6291-4191-b75d-e118e6108088 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:05.663832 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ba6a29-cbae-425b-ab3d-a216568b76cc None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:05.700706 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8188e1-4226-4f6e-af26-2b39b07754d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:05.701549 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8188e1-4226-4f6e-af26-2b39b07754d7 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:05.701835 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8188e1-4226-4f6e-af26-2b39b07754d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:05.702109 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8188e1-4226-4f6e-af26-2b39b07754d7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:05.713776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8188e1-4226-4f6e-af26-2b39b07754d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:06:05.721429 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-fa8188e1-4226-4f6e-af26-2b39b07754d7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 09:06:05.721429 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:05.721429 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:05.721429 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:05.721429 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:05.721429 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:05.721429 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:05.721429 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:05.721429 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:05.721429 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:05.721429 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:05.721429 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:05.721429 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:06:05.721429 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:06:05.721429 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:06:05.721429 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:06:05.722328 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:06:05.722328 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:06:05.722328 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:06:05.722328 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:06:05.722328 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:05.722328 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:05.722328 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:05.722328 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:05.722328 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:05.722328 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:05.722328 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:05.722328 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:05.722328 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:05.722328 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:05.722328 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:05.722328 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:05.723364 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:05.723364 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:05.723364 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:05.723364 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:05.723364 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:06:05.723364 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:06:05.723364 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:06:05.723364 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:06:05.723364 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:06:05.723364 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:06:05.723364 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 09:06:05.723364 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:05.723364 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 146/305] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:06:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:06:05.734038 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8188e1-4226-4f6e-af26-2b39b07754d7 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:05.771801 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d7e999-6291-4191-b75d-e118e6108088 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:05.789407 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:05.814354 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2dc58e-649b-4fea-bc5f-d66ce76fe7e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:05.815231 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2dc58e-649b-4fea-bc5f-d66ce76fe7e1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:05.815528 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2dc58e-649b-4fea-bc5f-d66ce76fe7e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:05.815787 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2dc58e-649b-4fea-bc5f-d66ce76fe7e1 None admin] PATH_INFO: `/v3/users/c410015dcf824f3397fbd9d0606bfa2c` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:05.817243 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2dc58e-649b-4fea-bc5f-d66ce76fe7e1 None admin] RBAC: Authorizing `identity:get_user(user_id=c410015dcf824f3397fbd9d0606bfa2c)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:05.818672 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2dc58e-649b-4fea-bc5f-d66ce76fe7e1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:05.820308 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 160/306] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 4 09:06:05 2022] GET /identity/v3/users/c410015dcf824f3397fbd9d0606bfa2c => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:06:05.822291 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d3334-e4a7-47ff-9992-d3bc88367d6f None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:05.822580 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d3334-e4a7-47ff-9992-d3bc88367d6f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:05.822861 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d3334-e4a7-47ff-9992-d3bc88367d6f None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:05.824223 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 161/307] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:05.829308 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869b8730-5d1e-42f6-92ce-78ab1ebe9c85 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:05.829673 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869b8730-5d1e-42f6-92ce-78ab1ebe9c85 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:05.829962 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869b8730-5d1e-42f6-92ce-78ab1ebe9c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:05.849682 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-869b8730-5d1e-42f6-92ce-78ab1ebe9c85 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:05.851956 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-869b8730-5d1e-42f6-92ce-78ab1ebe9c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:05.859715 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8188e1-4226-4f6e-af26-2b39b07754d7 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:05.871626 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:05.896861 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b9c0a-3eb9-4695-9110-3b4e98e6bcc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:05.897547 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b9c0a-3eb9-4695-9110-3b4e98e6bcc5 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:05.897795 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b9c0a-3eb9-4695-9110-3b4e98e6bcc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:05.898067 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b9c0a-3eb9-4695-9110-3b4e98e6bcc5 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:05.898811 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7b9c0a-3eb9-4695-9110-3b4e98e6bcc5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:05.899119 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7b9c0a-3eb9-4695-9110-3b4e98e6bcc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:05.900556 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7b9c0a-3eb9-4695-9110-3b4e98e6bcc5 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:05.910754 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 147/308] 10.222.0.71 () {56 vars in 1101 bytes} [Mon Jul 4 09:06:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:05.911409 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b9c0a-3eb9-4695-9110-3b4e98e6bcc5 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:05.930652 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 162/309] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:06:05.931909 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869b8730-5d1e-42f6-92ce-78ab1ebe9c85 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:06.021446 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:06.024848 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b9c0a-3eb9-4695-9110-3b4e98e6bcc5 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:06.047085 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869b8730-5d1e-42f6-92ce-78ab1ebe9c85 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:06.061231 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb757d64-7f03-4fbc-a0fa-3e048006d3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:06.062104 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb757d64-7f03-4fbc-a0fa-3e048006d3e5 None admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:06.062390 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb757d64-7f03-4fbc-a0fa-3e048006d3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:06.062669 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb757d64-7f03-4fbc-a0fa-3e048006d3e5 None admin] PATH_INFO: `/v3/system/users/c410015dcf824f3397fbd9d0606bfa2c/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:06.064164 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb757d64-7f03-4fbc-a0fa-3e048006d3e5 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=c410015dcf824f3397fbd9d0606bfa2c, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:06.065660 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb757d64-7f03-4fbc-a0fa-3e048006d3e5 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:06.075397 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 148/310] 10.222.0.71 () {58 vars in 1334 bytes} [Mon Jul 4 09:06:05 2022] PUT /identity/v3/system/users/c410015dcf824f3397fbd9d0606bfa2c/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:06:06.076286 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb757d64-7f03-4fbc-a0fa-3e048006d3e5 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:06.097344 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633c18da-2e20-4142-a7c5-f4fa3b0c6661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:06.098277 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c18da-2e20-4142-a7c5-f4fa3b0c6661 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:06.098653 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c18da-2e20-4142-a7c5-f4fa3b0c6661 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:06.098978 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c18da-2e20-4142-a7c5-f4fa3b0c6661 None admin] PATH_INFO: `/v3/users/384f8617e103485aa773cc65f1d4ca1e` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:06.114755 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633c18da-2e20-4142-a7c5-f4fa3b0c6661 None admin] RBAC: Authorizing `identity:get_user(user_id=384f8617e103485aa773cc65f1d4ca1e)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:06.116245 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633c18da-2e20-4142-a7c5-f4fa3b0c6661 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:06.119046 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 163/311] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 4 09:06:05 2022] GET /identity/v3/users/384f8617e103485aa773cc65f1d4ca1e => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:06.124945 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633c18da-2e20-4142-a7c5-f4fa3b0c6661 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:06.138831 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:06.214552 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb757d64-7f03-4fbc-a0fa-3e048006d3e5 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:06.256963 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906d8f67-4916-4e10-a79d-d5fd75b7d640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:06.258033 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d8f67-4916-4e10-a79d-d5fd75b7d640 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:06.258463 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d8f67-4916-4e10-a79d-d5fd75b7d640 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:06.258916 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d8f67-4916-4e10-a79d-d5fd75b7d640 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:06.268918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633c18da-2e20-4142-a7c5-f4fa3b0c6661 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:06.272865 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906d8f67-4916-4e10-a79d-d5fd75b7d640 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:06:06.281315 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-906d8f67-4916-4e10-a79d-d5fd75b7d640 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 09:06:06.281315 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:06.281315 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:06.281315 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:06.281315 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:06.281315 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:06.281315 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:06.281315 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:06.281315 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:06.281315 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:06.281315 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:06.281315 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:06.281315 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:06:06.281315 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:06:06.281315 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:06:06.281315 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:06:06.282215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:06:06.282215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:06:06.282215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:06:06.282215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:06:06.282215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:06.282215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:06.282215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:06.282215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:06.282215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:06.282215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:06.282215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:06.282215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:06.282215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:06.282215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:06.282215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:06.282215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:06.283014 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:06.283014 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:06.283014 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:06.283014 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:06.283014 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:06:06.283014 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:06:06.283014 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:06:06.283014 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:06:06.283014 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:06:06.283014 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:06:06.283014 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 09:06:06.283014 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:06.283014 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 149/312] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:06:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:06.290394 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906d8f67-4916-4e10-a79d-d5fd75b7d640 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:06.307325 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054fa4cb-35ae-4c11-a157-32a97bd63b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:06.308011 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054fa4cb-35ae-4c11-a157-32a97bd63b8b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:06.308293 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054fa4cb-35ae-4c11-a157-32a97bd63b8b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:06.308665 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054fa4cb-35ae-4c11-a157-32a97bd63b8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:06.309538 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054fa4cb-35ae-4c11-a157-32a97bd63b8b None admin] RBAC: Adding query filter params (role.id=56205a5df6514298a69fde673550f0aa, user.id=384f8617e103485aa773cc65f1d4ca1e, scope.system=all) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:06.309850 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054fa4cb-35ae-4c11-a157-32a97bd63b8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:06.311389 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054fa4cb-35ae-4c11-a157-32a97bd63b8b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:06.317256 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 164/313] 10.222.0.71 () {56 vars in 1308 bytes} [Mon Jul 4 09:06:06 2022] GET /identity/v3/role_assignments?user.id=384f8617e103485aa773cc65f1d4ca1e&scope.system=all&role.id=56205a5df6514298a69fde673550f0aa => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:06.328803 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:06.417895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906d8f67-4916-4e10-a79d-d5fd75b7d640 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:06.456264 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24957fa4-99b2-422e-95ea-d2bc4d40b7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:06.457361 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24957fa4-99b2-422e-95ea-d2bc4d40b7a8 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:06.457822 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24957fa4-99b2-422e-95ea-d2bc4d40b7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:06.458329 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24957fa4-99b2-422e-95ea-d2bc4d40b7a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:06.459272 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24957fa4-99b2-422e-95ea-d2bc4d40b7a8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:06.459772 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24957fa4-99b2-422e-95ea-d2bc4d40b7a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:06.461410 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24957fa4-99b2-422e-95ea-d2bc4d40b7a8 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:06.475928 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 150/314] 10.222.0.71 () {56 vars in 1101 bytes} [Mon Jul 4 09:06:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:06.482818 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054fa4cb-35ae-4c11-a157-32a97bd63b8b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:06.566590 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:06.593377 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054fa4cb-35ae-4c11-a157-32a97bd63b8b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:06.628722 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae9238f-9339-456e-b8b5-fed956462960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:06.630044 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae9238f-9339-456e-b8b5-fed956462960 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:06.630570 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae9238f-9339-456e-b8b5-fed956462960 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:06.631048 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae9238f-9339-456e-b8b5-fed956462960 None admin] PATH_INFO: `/v3/users/58287e2295c04e5f89626f1fc39bfb5d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:06.632706 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae9238f-9339-456e-b8b5-fed956462960 None admin] RBAC: Authorizing `identity:get_user(user_id=58287e2295c04e5f89626f1fc39bfb5d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:06.635034 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae9238f-9339-456e-b8b5-fed956462960 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:06.637463 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 165/315] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 4 09:06:06 2022] GET /identity/v3/users/58287e2295c04e5f89626f1fc39bfb5d => generated 297 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:06.643336 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24957fa4-99b2-422e-95ea-d2bc4d40b7a8 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:06.734062 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:06.748857 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24957fa4-99b2-422e-95ea-d2bc4d40b7a8 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:06.783652 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd446214-0911-4cbd-b7da-b0ee9ba1e9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:06.784308 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd446214-0911-4cbd-b7da-b0ee9ba1e9ba None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:06.784597 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd446214-0911-4cbd-b7da-b0ee9ba1e9ba None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:06.785065 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd446214-0911-4cbd-b7da-b0ee9ba1e9ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:06.859131 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd446214-0911-4cbd-b7da-b0ee9ba1e9ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:06.862379 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd446214-0911-4cbd-b7da-b0ee9ba1e9ba None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:06.947577 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:06.964424 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 151/316] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:06:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:06:06.969659 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae9238f-9339-456e-b8b5-fed956462960 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:07.074813 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae9238f-9339-456e-b8b5-fed956462960 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:07.110878 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed74017f-a361-4888-853b-01fbf54da057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:07.112112 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed74017f-a361-4888-853b-01fbf54da057 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:07.112595 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed74017f-a361-4888-853b-01fbf54da057 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:07.113129 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed74017f-a361-4888-853b-01fbf54da057 None admin] PATH_INFO: `/v3/projects/f8a33963b3504bf196b09f575de6e329` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:07.114539 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed74017f-a361-4888-853b-01fbf54da057 None admin] RBAC: Authorizing `identity:get_project(project_id=f8a33963b3504bf196b09f575de6e329)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:07.116196 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed74017f-a361-4888-853b-01fbf54da057 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:07.118529 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 166/317] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 4 09:06:06 2022] GET /identity/v3/projects/f8a33963b3504bf196b09f575de6e329 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:07.126483 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd446214-0911-4cbd-b7da-b0ee9ba1e9ba None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:07.224439 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd446214-0911-4cbd-b7da-b0ee9ba1e9ba None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:07.260469 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359f6cb1-1bd7-4e69-bc95-44cd017d4bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:07.261167 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359f6cb1-1bd7-4e69-bc95-44cd017d4bc0 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:07.261454 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359f6cb1-1bd7-4e69-bc95-44cd017d4bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:07.261733 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359f6cb1-1bd7-4e69-bc95-44cd017d4bc0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:07.262562 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359f6cb1-1bd7-4e69-bc95-44cd017d4bc0 None admin] RBAC: Adding query filter params (role.id=56205a5df6514298a69fde673550f0aa, scope.project.id=f8a33963b3504bf196b09f575de6e329, user.id=58287e2295c04e5f89626f1fc39bfb5d) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:07.262873 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359f6cb1-1bd7-4e69-bc95-44cd017d4bc0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:07.264324 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359f6cb1-1bd7-4e69-bc95-44cd017d4bc0 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:07.270907 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 152/318] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 4 09:06:07 2022] GET /identity/v3/role_assignments?user.id=58287e2295c04e5f89626f1fc39bfb5d&scope.project.id=f8a33963b3504bf196b09f575de6e329&role.id=56205a5df6514298a69fde673550f0aa => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:07.658062 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721b642-76c6-45b0-8d6b-43c1d7e9c0fd None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:07.658554 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721b642-76c6-45b0-8d6b-43c1d7e9c0fd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:07.658554 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721b642-76c6-45b0-8d6b-43c1d7e9c0fd None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:07.660174 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 167/319] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:07.665126 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a0ec65-8e22-4ef0-a509-ffd2099d755c None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:07.666045 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a0ec65-8e22-4ef0-a509-ffd2099d755c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:07.666394 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a0ec65-8e22-4ef0-a509-ffd2099d755c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:07.682994 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-24a0ec65-8e22-4ef0-a509-ffd2099d755c None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:07.684180 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-24a0ec65-8e22-4ef0-a509-ffd2099d755c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:07.730582 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:07.752320 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 153/320] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:06:07.775788 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e721b642-76c6-45b0-8d6b-43c1d7e9c0fd None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:07.787965 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:07.877515 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e721b642-76c6-45b0-8d6b-43c1d7e9c0fd None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:07.911753 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6660616-9bf5-4035-bb82-5307400c2b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:07.912479 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6660616-9bf5-4035-bb82-5307400c2b67 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:07.912749 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6660616-9bf5-4035-bb82-5307400c2b67 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:07.913020 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6660616-9bf5-4035-bb82-5307400c2b67 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:07.914200 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6660616-9bf5-4035-bb82-5307400c2b67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:07.915677 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6660616-9bf5-4035-bb82-5307400c2b67 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:07.917499 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 168/321] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:06:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:07.924177 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a0ec65-8e22-4ef0-a509-ffd2099d755c None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:08.026732 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a0ec65-8e22-4ef0-a509-ffd2099d755c None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:08.061028 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f04fee-2a00-4edb-8588-73030746d2e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:08.061728 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f04fee-2a00-4edb-8588-73030746d2e9 None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:08.062016 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f04fee-2a00-4edb-8588-73030746d2e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:08.062297 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f04fee-2a00-4edb-8588-73030746d2e9 None admin] PATH_INFO: `/v3/groups` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:08.062993 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f04fee-2a00-4edb-8588-73030746d2e9 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:08.064438 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f04fee-2a00-4edb-8588-73030746d2e9 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:08.080547 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 154/322] 10.222.0.71 () {60 vars in 1134 bytes} [Mon Jul 4 09:06:07 2022] POST /identity/v3/groups => generated 226 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:06:08.236929 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd428d1-cc5e-4ee1-b61d-20fb4b76e157 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:08.237257 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd428d1-cc5e-4ee1-b61d-20fb4b76e157 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:08.237543 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd428d1-cc5e-4ee1-b61d-20fb4b76e157 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:08.238922 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 169/323] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:08.244648 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee87a4a-ad92-4d3d-aaa7-5d12bcfd896c None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:08.245120 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee87a4a-ad92-4d3d-aaa7-5d12bcfd896c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:08.245305 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee87a4a-ad92-4d3d-aaa7-5d12bcfd896c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:08.262590 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-bee87a4a-ad92-4d3d-aaa7-5d12bcfd896c None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:08.263792 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-bee87a4a-ad92-4d3d-aaa7-5d12bcfd896c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:08.299044 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:08.338746 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 155/324] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:06:08.362557 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd428d1-cc5e-4ee1-b61d-20fb4b76e157 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:08.455269 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:08.474630 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65fd978-a9c5-4966-8909-2f0624e78111 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:08.474935 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65fd978-a9c5-4966-8909-2f0624e78111 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:08.475197 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65fd978-a9c5-4966-8909-2f0624e78111 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:08.476842 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 156/325] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:08.481623 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd428d1-cc5e-4ee1-b61d-20fb4b76e157 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:08.483331 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57ef7bf-3f12-45ee-b92d-15580f88c9a8 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:08.483627 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57ef7bf-3f12-45ee-b92d-15580f88c9a8 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:08.483913 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57ef7bf-3f12-45ee-b92d-15580f88c9a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:08.503495 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-d57ef7bf-3f12-45ee-b92d-15580f88c9a8 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:08.504615 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d57ef7bf-3f12-45ee-b92d-15580f88c9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:08.524360 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4102760-17f1-45bc-94bf-237cff29fae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:08.525993 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4102760-17f1-45bc-94bf-237cff29fae7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:08.526365 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4102760-17f1-45bc-94bf-237cff29fae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:08.526649 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4102760-17f1-45bc-94bf-237cff29fae7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:08.541284 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4102760-17f1-45bc-94bf-237cff29fae7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:06:08.550167 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-d4102760-17f1-45bc-94bf-237cff29fae7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:06:08.550167 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:08.550167 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:08.550167 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:08.550167 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:08.550167 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:08.550167 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:08.550167 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:08.550167 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:08.550167 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:08.550167 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:08.550167 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:08.550167 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:06:08.550167 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:06:08.550167 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:06:08.550167 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:06:08.551072 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:06:08.551072 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:06:08.551072 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:06:08.551072 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:06:08.551072 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:08.551072 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:08.551072 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:08.551072 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:08.551072 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:08.551072 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:08.551072 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:08.551072 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:08.551072 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:08.551072 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:08.551072 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:08.551072 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:08.552291 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:08.552291 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:08.552291 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:08.552291 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:08.552291 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:06:08.552291 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:06:08.552291 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:06:08.552291 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:06:08.552291 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:06:08.552291 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:06:08.552291 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:06:08.552291 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:08.552291 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 170/326] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:06:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:08.558123 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4102760-17f1-45bc-94bf-237cff29fae7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:08.588324 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 157/327] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:06:08.615593 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57ef7bf-3f12-45ee-b92d-15580f88c9a8 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:08.675640 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4102760-17f1-45bc-94bf-237cff29fae7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:08.714845 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6014859-4ca2-498c-b124-63ee490eb1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:08.715715 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6014859-4ca2-498c-b124-63ee490eb1b8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:08.715954 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6014859-4ca2-498c-b124-63ee490eb1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:08.716229 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6014859-4ca2-498c-b124-63ee490eb1b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:08.716982 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6014859-4ca2-498c-b124-63ee490eb1b8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:08.717320 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6014859-4ca2-498c-b124-63ee490eb1b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:08.718850 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6014859-4ca2-498c-b124-63ee490eb1b8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:08.724228 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:08.725982 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:08.727464 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57ef7bf-3f12-45ee-b92d-15580f88c9a8 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:08.731705 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 171/328] 10.222.0.71 () {56 vars in 1101 bytes} [Mon Jul 4 09:06:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:06:08.737482 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6014859-4ca2-498c-b124-63ee490eb1b8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:08.766834 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a23a140-7886-4b2e-ab77-975e324106fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:08.767717 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a23a140-7886-4b2e-ab77-975e324106fe None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:08.768109 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a23a140-7886-4b2e-ab77-975e324106fe None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:08.768564 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a23a140-7886-4b2e-ab77-975e324106fe None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:08.780312 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a23a140-7886-4b2e-ab77-975e324106fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:06:08.789521 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-3a23a140-7886-4b2e-ab77-975e324106fe None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 09:06:08.789521 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:08.789521 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:08.789521 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:08.789521 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:08.789521 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:08.789521 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:08.789521 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:08.789521 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:08.789521 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:08.789521 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:08.789521 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:08.789521 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:06:08.789521 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:06:08.789521 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:06:08.789521 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:06:08.791246 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:06:08.791246 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:06:08.791246 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:06:08.791246 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:06:08.791246 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:08.791246 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:08.791246 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:08.791246 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:08.791246 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:08.791246 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:08.791246 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:08.791246 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:08.791246 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:08.791246 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:08.791246 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:08.791246 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:08.792098 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:08.792098 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:08.792098 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:08.792098 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:08.792098 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:06:08.792098 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:06:08.792098 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:06:08.792098 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:06:08.792098 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:06:08.792098 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:06:08.792098 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 09:06:08.792098 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:08.798063 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 158/329] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:06:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:08.799303 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a23a140-7886-4b2e-ab77-975e324106fe None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:08.853694 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6014859-4ca2-498c-b124-63ee490eb1b8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:08.890664 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f482f75-3b64-48e2-852f-6ee311ad14ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:08.891926 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f482f75-3b64-48e2-852f-6ee311ad14ca None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:08.892410 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f482f75-3b64-48e2-852f-6ee311ad14ca None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:08.892890 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f482f75-3b64-48e2-852f-6ee311ad14ca None admin] PATH_INFO: `/v3/users/c410015dcf824f3397fbd9d0606bfa2c` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:08.894398 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f482f75-3b64-48e2-852f-6ee311ad14ca None admin] RBAC: Authorizing `identity:get_user(user_id=c410015dcf824f3397fbd9d0606bfa2c)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:08.896096 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f482f75-3b64-48e2-852f-6ee311ad14ca None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:08.898261 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 172/330] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 4 09:06:08 2022] GET /identity/v3/users/c410015dcf824f3397fbd9d0606bfa2c => generated 293 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:08.904451 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f482f75-3b64-48e2-852f-6ee311ad14ca None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:08.911145 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a23a140-7886-4b2e-ab77-975e324106fe None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:08.949095 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbba55e2-8bc9-46ef-a8c8-4b2df67dbb91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:08.949834 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba55e2-8bc9-46ef-a8c8-4b2df67dbb91 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:08.950121 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba55e2-8bc9-46ef-a8c8-4b2df67dbb91 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:08.950401 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba55e2-8bc9-46ef-a8c8-4b2df67dbb91 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:08.951158 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbba55e2-8bc9-46ef-a8c8-4b2df67dbb91 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:08.951468 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbba55e2-8bc9-46ef-a8c8-4b2df67dbb91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:08.952977 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbba55e2-8bc9-46ef-a8c8-4b2df67dbb91 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:08.963110 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 159/331] 10.222.0.71 () {56 vars in 1101 bytes} [Mon Jul 4 09:06:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:08.969105 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbba55e2-8bc9-46ef-a8c8-4b2df67dbb91 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:08.981077 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:09.010475 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:09.018940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f482f75-3b64-48e2-852f-6ee311ad14ca None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:09.058073 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864d68fe-c0e2-45d4-b681-faf95c9e463b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:09.058635 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864d68fe-c0e2-45d4-b681-faf95c9e463b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:09.059020 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864d68fe-c0e2-45d4-b681-faf95c9e463b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:09.059400 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864d68fe-c0e2-45d4-b681-faf95c9e463b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:09.060597 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864d68fe-c0e2-45d4-b681-faf95c9e463b None admin] RBAC: Adding query filter params (role.id=8692e99e9463459f9b7caa96b3051b93, user.id=c410015dcf824f3397fbd9d0606bfa2c, scope.system=all) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:09.060825 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864d68fe-c0e2-45d4-b681-faf95c9e463b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:09.062432 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864d68fe-c0e2-45d4-b681-faf95c9e463b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:09.077416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 173/332] 10.222.0.71 () {56 vars in 1308 bytes} [Mon Jul 4 09:06:08 2022] GET /identity/v3/role_assignments?user.id=c410015dcf824f3397fbd9d0606bfa2c&scope.system=all&role.id=8692e99e9463459f9b7caa96b3051b93 => generated 510 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:09.087377 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbba55e2-8bc9-46ef-a8c8-4b2df67dbb91 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:09.131681 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6695a44c-cf9c-402c-9a9b-b120c5a37194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:09.132866 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695a44c-cf9c-402c-9a9b-b120c5a37194 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:09.133318 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695a44c-cf9c-402c-9a9b-b120c5a37194 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:09.133737 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695a44c-cf9c-402c-9a9b-b120c5a37194 None admin] PATH_INFO: `/v3/users/384f8617e103485aa773cc65f1d4ca1e` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:09.137887 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6695a44c-cf9c-402c-9a9b-b120c5a37194 None admin] RBAC: Authorizing `identity:get_user(user_id=384f8617e103485aa773cc65f1d4ca1e)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:09.139506 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6695a44c-cf9c-402c-9a9b-b120c5a37194 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:09.141644 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 160/333] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 4 09:06:08 2022] GET /identity/v3/users/384f8617e103485aa773cc65f1d4ca1e => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:06:09.147483 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864d68fe-c0e2-45d4-b681-faf95c9e463b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:09.263868 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864d68fe-c0e2-45d4-b681-faf95c9e463b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:09.291149 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:09.300012 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f445986-3cce-4bf7-a1cb-bcd0d7302076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:09.300896 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f445986-3cce-4bf7-a1cb-bcd0d7302076 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:09.301217 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f445986-3cce-4bf7-a1cb-bcd0d7302076 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:09.301498 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f445986-3cce-4bf7-a1cb-bcd0d7302076 None admin] PATH_INFO: `/v3/system/users/384f8617e103485aa773cc65f1d4ca1e/roles/56205a5df6514298a69fde673550f0aa` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:09.303002 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f445986-3cce-4bf7-a1cb-bcd0d7302076 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=384f8617e103485aa773cc65f1d4ca1e, role_id=56205a5df6514298a69fde673550f0aa)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:09.304382 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f445986-3cce-4bf7-a1cb-bcd0d7302076 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:09.312953 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 174/334] 10.222.0.71 () {58 vars in 1334 bytes} [Mon Jul 4 09:06:09 2022] PUT /identity/v3/system/users/384f8617e103485aa773cc65f1d4ca1e/roles/56205a5df6514298a69fde673550f0aa => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:06:10.661767 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f114cf2-3825-4ad7-8574-642a40477202 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:10.662261 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f114cf2-3825-4ad7-8574-642a40477202 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:10.662261 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f114cf2-3825-4ad7-8574-642a40477202 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:10.663892 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 161/335] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:10.669837 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c0753-1d4a-409c-9fdd-81ec738aeea7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:10.670289 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c0753-1d4a-409c-9fdd-81ec738aeea7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:10.670470 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c0753-1d4a-409c-9fdd-81ec738aeea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:10.695730 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7d8c0753-1d4a-409c-9fdd-81ec738aeea7 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:10.697523 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7d8c0753-1d4a-409c-9fdd-81ec738aeea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:10.773143 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 175/336] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:06:10.808805 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f114cf2-3825-4ad7-8574-642a40477202 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:10.895485 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caeb420-90a3-4bab-9afc-abca9a31eab8 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:10.895810 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caeb420-90a3-4bab-9afc-abca9a31eab8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:10.896054 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caeb420-90a3-4bab-9afc-abca9a31eab8 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:10.897463 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 176/337] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:10.905515 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c13ee-011c-478c-b60f-4007bd3a3fc6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:10.905515 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c13ee-011c-478c-b60f-4007bd3a3fc6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:10.905725 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c13ee-011c-478c-b60f-4007bd3a3fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:10.925992 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-df0c13ee-011c-478c-b60f-4007bd3a3fc6 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:10.925992 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-df0c13ee-011c-478c-b60f-4007bd3a3fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:10.932301 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:10.957064 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f114cf2-3825-4ad7-8574-642a40477202 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:11.019168 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9ea248-e7cc-4c3f-b727-d456fac3eac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:11.019986 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ea248-e7cc-4c3f-b727-d456fac3eac7 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:11.020273 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ea248-e7cc-4c3f-b727-d456fac3eac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:11.020553 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ea248-e7cc-4c3f-b727-d456fac3eac7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:11.041654 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9ea248-e7cc-4c3f-b727-d456fac3eac7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:06:11.049890 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-fd9ea248-e7cc-4c3f-b727-d456fac3eac7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:06:11.049890 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:11.049890 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:11.049890 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:11.049890 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:11.049890 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:11.049890 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:11.049890 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:11.049890 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:11.049890 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:11.049890 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:11.049890 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:11.049890 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:06:11.049890 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:06:11.049890 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:06:11.049890 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:06:11.050774 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:06:11.050774 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:06:11.050774 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:06:11.050774 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:06:11.050774 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:11.050774 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:11.050774 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:11.050774 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:11.050774 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:11.050774 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:11.050774 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:11.050774 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:11.050774 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:11.050774 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:11.050774 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:11.050774 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:11.051563 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:11.051563 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:11.051563 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:11.051563 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:11.051563 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:06:11.051563 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:06:11.051563 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:06:11.051563 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:06:11.051563 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:06:11.051563 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:06:11.051563 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:06:11.051563 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:11.061314 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 162/338] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:06:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:11.062313 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9ea248-e7cc-4c3f-b727-d456fac3eac7 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:11.070946 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 177/339] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:06:11.162408 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0c13ee-011c-478c-b60f-4007bd3a3fc6 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:11.210712 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9ea248-e7cc-4c3f-b727-d456fac3eac7 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:11.249242 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd697dc6-1cde-4049-8a5d-5d5f261984c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:11.250264 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd697dc6-1cde-4049-8a5d-5d5f261984c4 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:11.250744 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd697dc6-1cde-4049-8a5d-5d5f261984c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:11.251216 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd697dc6-1cde-4049-8a5d-5d5f261984c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:11.252161 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd697dc6-1cde-4049-8a5d-5d5f261984c4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:11.252717 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd697dc6-1cde-4049-8a5d-5d5f261984c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:11.254326 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd697dc6-1cde-4049-8a5d-5d5f261984c4 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:11.264767 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 163/340] 10.222.0.71 () {56 vars in 1101 bytes} [Mon Jul 4 09:06:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:11.274013 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd697dc6-1cde-4049-8a5d-5d5f261984c4 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:11.297410 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:11.297629 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:11.427620 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd697dc6-1cde-4049-8a5d-5d5f261984c4 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:11.444637 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0c13ee-011c-478c-b60f-4007bd3a3fc6 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:11.468451 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc708b-abf7-432f-8293-f1d6c9a7c740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:11.469081 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc708b-abf7-432f-8293-f1d6c9a7c740 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:11.469389 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc708b-abf7-432f-8293-f1d6c9a7c740 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:11.469668 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc708b-abf7-432f-8293-f1d6c9a7c740 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:11.526208 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a39c5f7-0cc2-4cfa-8135-6b6c5f682e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:11.527041 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a39c5f7-0cc2-4cfa-8135-6b6c5f682e79 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:11.527331 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a39c5f7-0cc2-4cfa-8135-6b6c5f682e79 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:11.527613 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a39c5f7-0cc2-4cfa-8135-6b6c5f682e79 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:11.550987 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bc708b-abf7-432f-8293-f1d6c9a7c740 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:11.552553 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bc708b-abf7-432f-8293-f1d6c9a7c740 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:11.562118 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a39c5f7-0cc2-4cfa-8135-6b6c5f682e79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:06:11.569525 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:11.575132 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-0a39c5f7-0cc2-4cfa-8135-6b6c5f682e79 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:06:11.575132 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:11.575132 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:11.575132 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:11.575132 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:11.575132 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:11.575132 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:11.575132 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:11.575132 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:11.575132 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:11.575132 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:11.575132 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:11.575132 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:06:11.575132 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:06:11.575132 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:06:11.575132 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:06:11.576041 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:06:11.576041 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:06:11.576041 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:06:11.576041 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:06:11.576041 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:11.576041 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:11.576041 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:11.576041 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:11.576041 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:11.576041 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:11.576041 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:11.576041 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:11.576041 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:11.576041 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:11.576041 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:11.576041 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:11.577416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:11.577416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:11.577416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:11.577416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:11.577416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:06:11.577416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:06:11.577416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:06:11.577416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:06:11.577416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:06:11.577416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:06:11.577416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:06:11.577416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:11.577416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 178/341] 10.222.0.71 () {56 vars in 1114 bytes} [Mon Jul 4 09:06:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 415 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:06:11.578071 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b9667d-6eef-4717-a4ca-2a63e6f29f71 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:11.578071 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b9667d-6eef-4717-a4ca-2a63e6f29f71 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:11.578432 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b9667d-6eef-4717-a4ca-2a63e6f29f71 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:11.579872 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 179/342] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:11.585639 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b9667d-6eef-4717-a4ca-2a63e6f29f71 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:11.670316 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 164/343] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:06:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:06:11.671731 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ec6a8-7daf-4faf-b875-a6ff35eb6fd3 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:11.671976 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ec6a8-7daf-4faf-b875-a6ff35eb6fd3 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:11.672310 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ec6a8-7daf-4faf-b875-a6ff35eb6fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:11.697491 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-d12ec6a8-7daf-4faf-b875-a6ff35eb6fd3 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:11.698693 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d12ec6a8-7daf-4faf-b875-a6ff35eb6fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:11.811398 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 165/344] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:06:11.812427 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12ec6a8-7daf-4faf-b875-a6ff35eb6fd3 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:11.856946 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b9667d-6eef-4717-a4ca-2a63e6f29f71 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:11.875432 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:11.898943 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:11.920201 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e9086-5ed5-454f-96b2-b7a00232b005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:11.920878 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e9086-5ed5-454f-96b2-b7a00232b005 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:11.921199 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e9086-5ed5-454f-96b2-b7a00232b005 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:11.921485 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e9086-5ed5-454f-96b2-b7a00232b005 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:11.922391 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898e9086-5ed5-454f-96b2-b7a00232b005 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:11.922711 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898e9086-5ed5-454f-96b2-b7a00232b005 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:11.926422 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898e9086-5ed5-454f-96b2-b7a00232b005 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:11.938705 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 180/345] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 4 09:06:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:11.942420 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e9086-5ed5-454f-96b2-b7a00232b005 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:12.043054 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12ec6a8-7daf-4faf-b875-a6ff35eb6fd3 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:12.070494 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e9086-5ed5-454f-96b2-b7a00232b005 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:12.112912 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d4891-d639-4922-9937-f9e20402602d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:12.114161 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d4891-d639-4922-9937-f9e20402602d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:12.114869 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d4891-d639-4922-9937-f9e20402602d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:12.115319 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d4891-d639-4922-9937-f9e20402602d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:12.127565 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d4891-d639-4922-9937-f9e20402602d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:06:12.137160 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-2c4d4891-d639-4922-9937-f9e20402602d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:06:12.137160 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:12.137160 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:12.137160 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:12.137160 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:12.137160 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:12.137160 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:12.137160 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:12.137160 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:12.137160 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:12.137160 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:12.137160 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:12.137160 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:06:12.137160 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:06:12.137160 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:06:12.137160 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:06:12.138069 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:06:12.138069 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:06:12.138069 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:06:12.138069 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:06:12.138069 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:12.138069 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:12.138069 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:12.138069 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:12.138069 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:12.138069 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:12.138069 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:12.138069 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:12.138069 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:12.138069 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:12.138069 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:12.138069 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:12.138979 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:12.138979 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:12.138979 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:12.138979 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:12.138979 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:06:12.138979 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:06:12.138979 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:06:12.138979 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:06:12.138979 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:06:12.138979 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:06:12.138979 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:06:12.138979 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:12.142645 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 181/346] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:06:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:12.142891 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893cce93-6ad1-4ff9-8325-52f99ec17f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:12.143826 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d4891-d639-4922-9937-f9e20402602d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:12.144086 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893cce93-6ad1-4ff9-8325-52f99ec17f43 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:12.144380 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893cce93-6ad1-4ff9-8325-52f99ec17f43 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:12.144674 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893cce93-6ad1-4ff9-8325-52f99ec17f43 None admin] PATH_INFO: `/v3/projects/e192ac889bad428985088eea5ce750ab` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:12.148706 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893cce93-6ad1-4ff9-8325-52f99ec17f43 None admin] RBAC: Authorizing `identity:get_project(project_id=e192ac889bad428985088eea5ce750ab)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:12.150296 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893cce93-6ad1-4ff9-8325-52f99ec17f43 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:12.152252 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 166/347] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 4 09:06:11 2022] GET /identity/v3/projects/e192ac889bad428985088eea5ce750ab => generated 298 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:12.157984 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893cce93-6ad1-4ff9-8325-52f99ec17f43 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:12.208160 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:12.276644 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d4891-d639-4922-9937-f9e20402602d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:12.326266 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d93d05c-195d-47f7-80b6-5b04207b856a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:12.327008 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93d05c-195d-47f7-80b6-5b04207b856a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:12.327427 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93d05c-195d-47f7-80b6-5b04207b856a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:12.328019 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93d05c-195d-47f7-80b6-5b04207b856a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:12.366700 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893cce93-6ad1-4ff9-8325-52f99ec17f43 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:12.400365 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:12.418098 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d93d05c-195d-47f7-80b6-5b04207b856a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:12.420110 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d93d05c-195d-47f7-80b6-5b04207b856a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:12.438109 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0731356-376a-4ef9-a723-f2919b99d13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:12.439142 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0731356-376a-4ef9-a723-f2919b99d13c None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:12.440593 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0731356-376a-4ef9-a723-f2919b99d13c None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:12.441129 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0731356-376a-4ef9-a723-f2919b99d13c None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:12.442812 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0731356-376a-4ef9-a723-f2919b99d13c None admin] RBAC: Adding query filter params (name=member) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:12.443505 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0731356-376a-4ef9-a723-f2919b99d13c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:12.447584 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0731356-376a-4ef9-a723-f2919b99d13c None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:12.464641 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 167/348] 10.222.0.71 () {56 vars in 1101 bytes} [Mon Jul 4 09:06:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:12.467256 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0731356-376a-4ef9-a723-f2919b99d13c None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:12.485402 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:12.611769 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 182/349] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:06:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:06:12.614429 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d93d05c-195d-47f7-80b6-5b04207b856a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:12.663980 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0731356-376a-4ef9-a723-f2919b99d13c None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:12.724792 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6303d84d-3dae-4287-ac18-d70598bca646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:12.725903 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303d84d-3dae-4287-ac18-d70598bca646 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:12.726192 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303d84d-3dae-4287-ac18-d70598bca646 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:12.726474 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303d84d-3dae-4287-ac18-d70598bca646 None admin] PATH_INFO: `/v3/groups/6e50e4b8168340a8bc57713559ff23ae` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:12.736964 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6303d84d-3dae-4287-ac18-d70598bca646 None admin] RBAC: Authorizing `identity:get_group(group_id=6e50e4b8168340a8bc57713559ff23ae)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:12.738543 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6303d84d-3dae-4287-ac18-d70598bca646 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:12.740449 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 168/350] 10.222.0.71 () {56 vars in 1180 bytes} [Mon Jul 4 09:06:12 2022] GET /identity/v3/groups/6e50e4b8168340a8bc57713559ff23ae => generated 226 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:12.741192 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6303d84d-3dae-4287-ac18-d70598bca646 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:12.787582 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d93d05c-195d-47f7-80b6-5b04207b856a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:12.792828 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:12.828995 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:12.835981 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f5e477-6c9f-4174-afc0-a59a7ed01353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:12.836650 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5e477-6c9f-4174-afc0-a59a7ed01353 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:12.836970 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5e477-6c9f-4174-afc0-a59a7ed01353 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:12.838100 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5e477-6c9f-4174-afc0-a59a7ed01353 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:12.866171 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6303d84d-3dae-4287-ac18-d70598bca646 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:12.912153 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18287903-ae11-48bf-a1fc-69eba791966b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:12.913202 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18287903-ae11-48bf-a1fc-69eba791966b None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:12.913464 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18287903-ae11-48bf-a1fc-69eba791966b None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:12.913750 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18287903-ae11-48bf-a1fc-69eba791966b None admin] PATH_INFO: `/v3/projects/e192ac889bad428985088eea5ce750ab` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:12.915869 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18287903-ae11-48bf-a1fc-69eba791966b None admin] RBAC: Authorizing `identity:get_project(project_id=e192ac889bad428985088eea5ce750ab)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:12.917399 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18287903-ae11-48bf-a1fc-69eba791966b None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:12.919202 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 169/351] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 4 09:06:12 2022] GET /identity/v3/projects/e192ac889bad428985088eea5ce750ab => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:12.923141 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76893249-f8fd-4546-821c-41f6fcb67e19 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:12.923578 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76893249-f8fd-4546-821c-41f6fcb67e19 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:12.924194 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76893249-f8fd-4546-821c-41f6fcb67e19 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:12.925752 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 170/352] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:12.927658 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193478a3-978b-41f0-8d81-59072a6bcf8f None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:12.928875 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193478a3-978b-41f0-8d81-59072a6bcf8f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:12.928875 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193478a3-978b-41f0-8d81-59072a6bcf8f None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:12.931724 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 171/353] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:12.931724 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193478a3-978b-41f0-8d81-59072a6bcf8f None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:12.938802 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f5e477-6c9f-4174-afc0-a59a7ed01353 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:12.940270 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f5e477-6c9f-4174-afc0-a59a7ed01353 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:13.044918 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193478a3-978b-41f0-8d81-59072a6bcf8f None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:13.047493 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:13.064956 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 183/354] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:06:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:06:13.069068 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f5e477-6c9f-4174-afc0-a59a7ed01353 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:13.093533 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870aba0b-5b4f-4115-9f4b-12d78a682d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:13.094208 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870aba0b-5b4f-4115-9f4b-12d78a682d2f None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:13.094498 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870aba0b-5b4f-4115-9f4b-12d78a682d2f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:13.094783 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870aba0b-5b4f-4115-9f4b-12d78a682d2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:13.095627 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870aba0b-5b4f-4115-9f4b-12d78a682d2f None admin] RBAC: Adding query filter params (group.id=6e50e4b8168340a8bc57713559ff23ae, role.id=8692e99e9463459f9b7caa96b3051b93, scope.project.id=e192ac889bad428985088eea5ce750ab) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:13.095946 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870aba0b-5b4f-4115-9f4b-12d78a682d2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:13.097475 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870aba0b-5b4f-4115-9f4b-12d78a682d2f None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:13.101023 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:13.108191 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 172/355] 10.222.0.71 () {56 vars in 1376 bytes} [Mon Jul 4 09:06:12 2022] GET /identity/v3/role_assignments?group.id=6e50e4b8168340a8bc57713559ff23ae&scope.project.id=e192ac889bad428985088eea5ce750ab&role.id=8692e99e9463459f9b7caa96b3051b93 => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:13.109833 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6656f0-4f8d-47e0-960d-e576c0185bb0 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:13.110131 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6656f0-4f8d-47e0-960d-e576c0185bb0 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:13.110413 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6656f0-4f8d-47e0-960d-e576c0185bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:13.135709 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-ef6656f0-4f8d-47e0-960d-e576c0185bb0 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:13.137811 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ef6656f0-4f8d-47e0-960d-e576c0185bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:13.207562 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f5e477-6c9f-4174-afc0-a59a7ed01353 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:13.236074 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 173/356] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:06:13.238218 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9b1dd7-4ec9-4d79-9317-50bc82e248a8 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:13.238974 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9b1dd7-4ec9-4d79-9317-50bc82e248a8 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:13.239481 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9b1dd7-4ec9-4d79-9317-50bc82e248a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:13.252025 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a851f9b4-032f-40f2-96d0-ebe40a891167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:13.252962 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a851f9b4-032f-40f2-96d0-ebe40a891167 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:13.253284 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a851f9b4-032f-40f2-96d0-ebe40a891167 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:13.253624 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a851f9b4-032f-40f2-96d0-ebe40a891167 None admin] PATH_INFO: `/v3/groups/6e50e4b8168340a8bc57713559ff23ae` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:13.254869 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a851f9b4-032f-40f2-96d0-ebe40a891167 None admin] RBAC: Authorizing `identity:get_group(group_id=6e50e4b8168340a8bc57713559ff23ae)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:13.256582 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a851f9b4-032f-40f2-96d0-ebe40a891167 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:13.258903 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 184/357] 10.222.0.71 () {56 vars in 1180 bytes} [Mon Jul 4 09:06:13 2022] GET /identity/v3/groups/6e50e4b8168340a8bc57713559ff23ae => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:13.260282 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6989af-e257-474d-a4d4-3f949d44e16b None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:13.260577 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6989af-e257-474d-a4d4-3f949d44e16b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:13.260860 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6989af-e257-474d-a4d4-3f949d44e16b None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:13.262430 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 185/358] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:13.263422 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6989af-e257-474d-a4d4-3f949d44e16b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:13.263630 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-fc9b1dd7-4ec9-4d79-9317-50bc82e248a8 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:13.265371 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-fc9b1dd7-4ec9-4d79-9317-50bc82e248a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:13.354278 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 174/359] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:06:13.355206 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9b1dd7-4ec9-4d79-9317-50bc82e248a8 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:13.397341 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:13.410673 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:13.420241 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6989af-e257-474d-a4d4-3f949d44e16b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:13.468241 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5928d9-40b1-49cd-ba99-968f772e7bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:13.469176 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5928d9-40b1-49cd-ba99-968f772e7bb0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:13.469465 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5928d9-40b1-49cd-ba99-968f772e7bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:13.469746 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5928d9-40b1-49cd-ba99-968f772e7bb0 None admin] PATH_INFO: `/v3/projects/f8a33963b3504bf196b09f575de6e329` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:13.470947 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5928d9-40b1-49cd-ba99-968f772e7bb0 None admin] RBAC: Authorizing `identity:get_project(project_id=f8a33963b3504bf196b09f575de6e329)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:13.472396 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5928d9-40b1-49cd-ba99-968f772e7bb0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:13.475279 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 186/360] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 4 09:06:13 2022] GET /identity/v3/projects/f8a33963b3504bf196b09f575de6e329 => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:13.478831 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a276f-6d52-4c0f-b043-f54caebb4059 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:13.478831 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a276f-6d52-4c0f-b043-f54caebb4059 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:13.478831 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a276f-6d52-4c0f-b043-f54caebb4059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:13.504274 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f72a276f-6d52-4c0f-b043-f54caebb4059 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:13.505528 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f72a276f-6d52-4c0f-b043-f54caebb4059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:13.506741 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9b1dd7-4ec9-4d79-9317-50bc82e248a8 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:13.549728 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13f0758-ba75-455e-8c24-b02ba624c4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:13.550773 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f0758-ba75-455e-8c24-b02ba624c4cd None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:13.551135 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f0758-ba75-455e-8c24-b02ba624c4cd None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:13.551490 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f0758-ba75-455e-8c24-b02ba624c4cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:13.552409 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13f0758-ba75-455e-8c24-b02ba624c4cd None admin] RBAC: Adding query filter params (group.id=6e50e4b8168340a8bc57713559ff23ae, role.id=5fb58444d5604bf689014e27dccacb45, scope.project.id=e192ac889bad428985088eea5ce750ab) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:13.552796 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13f0758-ba75-455e-8c24-b02ba624c4cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:13.554370 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13f0758-ba75-455e-8c24-b02ba624c4cd None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:13.562390 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 175/361] 10.222.0.71 () {56 vars in 1376 bytes} [Mon Jul 4 09:06:13 2022] GET /identity/v3/role_assignments?group.id=6e50e4b8168340a8bc57713559ff23ae&scope.project.id=e192ac889bad428985088eea5ce750ab&role.id=5fb58444d5604bf689014e27dccacb45 => generated 260 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:13.565129 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13f0758-ba75-455e-8c24-b02ba624c4cd None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:13.609654 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 187/362] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:06:13.610860 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72a276f-6d52-4c0f-b043-f54caebb4059 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:13.700407 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:13.710378 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13f0758-ba75-455e-8c24-b02ba624c4cd None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:13.727474 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:13.733266 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72a276f-6d52-4c0f-b043-f54caebb4059 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:13.764626 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c6e56-402c-46a8-8e8b-dcfc8e4f9ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:13.765545 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c6e56-402c-46a8-8e8b-dcfc8e4f9ae1 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:13.765889 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c6e56-402c-46a8-8e8b-dcfc8e4f9ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:13.766170 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c6e56-402c-46a8-8e8b-dcfc8e4f9ae1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:13.773888 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf5e8d2-d41f-492f-b3c6-ae6b6a2d6f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:13.775640 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf5e8d2-d41f-492f-b3c6-ae6b6a2d6f7a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:13.776142 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf5e8d2-d41f-492f-b3c6-ae6b6a2d6f7a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:13.776627 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf5e8d2-d41f-492f-b3c6-ae6b6a2d6f7a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:13.782271 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29c6e56-402c-46a8-8e8b-dcfc8e4f9ae1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:06:13.788881 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf5e8d2-d41f-492f-b3c6-ae6b6a2d6f7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:06:13.791222 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-d29c6e56-402c-46a8-8e8b-dcfc8e4f9ae1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:06:13.791222 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:13.791222 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:13.791222 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:13.791222 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:13.791222 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:13.791222 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:13.791222 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:13.791222 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:13.791222 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:13.791222 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:13.791222 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:13.791222 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:06:13.791222 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:06:13.791222 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:06:13.791222 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:06:13.792155 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:06:13.792155 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:06:13.792155 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:06:13.792155 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:06:13.792155 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:13.792155 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:13.792155 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:13.792155 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:13.792155 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:13.792155 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:13.792155 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:13.792155 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:13.792155 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:13.792155 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:13.792155 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:13.792155 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:13.793005 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:13.793005 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:13.793005 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:13.793005 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:13.793005 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:06:13.793005 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:06:13.793005 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:06:13.793005 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:06:13.793005 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:06:13.793005 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:06:13.793005 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:06:13.793005 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:13.795908 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 176/363] 10.222.0.71 () {56 vars in 1114 bytes} [Mon Jul 4 09:06:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:13.797497 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c6e56-402c-46a8-8e8b-dcfc8e4f9ae1 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:13.801490 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-adf5e8d2-d41f-492f-b3c6-ae6b6a2d6f7a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 09:06:13.801490 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:13.801490 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:13.801490 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:13.801490 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:13.801490 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:13.801490 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:13.801490 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:13.801490 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:13.801490 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:13.801490 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:13.801490 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:13.801490 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:06:13.801490 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:06:13.801490 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:06:13.801490 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:06:13.802416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:06:13.802416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:06:13.802416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:06:13.802416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:06:13.802416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:13.802416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:13.802416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:13.802416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:13.802416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:13.802416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:13.802416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:13.802416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:13.802416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:13.802416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:13.802416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:13.802416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:13.803241 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:13.803241 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:13.803241 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:13.803241 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:13.803241 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:06:13.803241 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:06:13.803241 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:06:13.803241 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:06:13.803241 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:06:13.803241 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:06:13.803241 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 09:06:13.803241 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:13.805760 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 188/364] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:06:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:13.806802 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf5e8d2-d41f-492f-b3c6-ae6b6a2d6f7a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:13.916209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c6e56-402c-46a8-8e8b-dcfc8e4f9ae1 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:13.921002 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf5e8d2-d41f-492f-b3c6-ae6b6a2d6f7a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:13.955425 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93e9878-90b5-4f33-b75c-41ce07f97941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:13.956416 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93e9878-90b5-4f33-b75c-41ce07f97941 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:13.956774 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93e9878-90b5-4f33-b75c-41ce07f97941 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:13.957082 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93e9878-90b5-4f33-b75c-41ce07f97941 None admin] PATH_INFO: `/v3/groups/6e50e4b8168340a8bc57713559ff23ae` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:13.958464 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93e9878-90b5-4f33-b75c-41ce07f97941 None admin] RBAC: Authorizing `identity:get_group(group_id=6e50e4b8168340a8bc57713559ff23ae)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:13.958808 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfab010a-03e3-4599-811e-2cad49157203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:13.960019 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93e9878-90b5-4f33-b75c-41ce07f97941 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:13.960381 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab010a-03e3-4599-811e-2cad49157203 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:13.960855 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab010a-03e3-4599-811e-2cad49157203 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:13.961361 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab010a-03e3-4599-811e-2cad49157203 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:13.962817 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 177/365] 10.222.0.71 () {56 vars in 1180 bytes} [Mon Jul 4 09:06:13 2022] GET /identity/v3/groups/6e50e4b8168340a8bc57713559ff23ae => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:13.963746 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93e9878-90b5-4f33-b75c-41ce07f97941 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:13.976396 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfab010a-03e3-4599-811e-2cad49157203 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:06:13.985414 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-bfab010a-03e3-4599-811e-2cad49157203 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:06:13.985414 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:13.985414 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:13.985414 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:13.985414 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:13.985414 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:13.985414 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:13.985414 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:13.985414 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:13.985414 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:13.985414 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:13.985414 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:13.985414 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:06:13.985414 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:06:13.985414 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:06:13.985414 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:06:13.986318 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:06:13.986318 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:06:13.986318 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:06:13.986318 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:06:13.986318 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:13.986318 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:13.986318 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:13.986318 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:13.986318 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:13.986318 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:13.986318 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:13.986318 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:13.986318 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:13.986318 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:13.986318 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:13.986318 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:13.987115 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:13.987115 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:13.987115 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:13.987115 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:13.987115 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:06:13.987115 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:06:13.987115 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:06:13.987115 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:06:13.987115 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:06:13.987115 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:06:13.987115 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:06:13.987115 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:13.989675 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 189/366] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 4 09:06:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:13.990732 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfab010a-03e3-4599-811e-2cad49157203 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:13.991723 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:14.013314 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:14.081476 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93e9878-90b5-4f33-b75c-41ce07f97941 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:14.119549 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfab010a-03e3-4599-811e-2cad49157203 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:14.122662 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911b41c0-2270-4e9f-ad38-28365939e226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:14.123608 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b41c0-2270-4e9f-ad38-28365939e226 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:14.124114 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b41c0-2270-4e9f-ad38-28365939e226 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:14.124601 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b41c0-2270-4e9f-ad38-28365939e226 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:14.125652 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911b41c0-2270-4e9f-ad38-28365939e226 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:14.126173 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911b41c0-2270-4e9f-ad38-28365939e226 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:14.128242 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911b41c0-2270-4e9f-ad38-28365939e226 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:14.137437 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 178/367] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 4 09:06:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:14.138572 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911b41c0-2270-4e9f-ad38-28365939e226 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:14.174726 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952c162e-c930-42c1-949d-8ffbd4559a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:14.175847 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c162e-c930-42c1-949d-8ffbd4559a20 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:14.176341 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c162e-c930-42c1-949d-8ffbd4559a20 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:14.177782 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c162e-c930-42c1-949d-8ffbd4559a20 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:14.178777 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952c162e-c930-42c1-949d-8ffbd4559a20 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:14.180434 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952c162e-c930-42c1-949d-8ffbd4559a20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:14.182292 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952c162e-c930-42c1-949d-8ffbd4559a20 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:14.191586 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 190/368] 10.222.0.71 () {56 vars in 1101 bytes} [Mon Jul 4 09:06:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:14.192737 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952c162e-c930-42c1-949d-8ffbd4559a20 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:14.247689 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:14.261833 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911b41c0-2270-4e9f-ad38-28365939e226 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:14.301384 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:14.307971 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd9543f-d5de-4702-b81a-e6abdc932fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:14.308702 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9543f-d5de-4702-b81a-e6abdc932fd8 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:14.308956 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9543f-d5de-4702-b81a-e6abdc932fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:14.309274 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9543f-d5de-4702-b81a-e6abdc932fd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:14.310145 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd9543f-d5de-4702-b81a-e6abdc932fd8 None admin] RBAC: Adding query filter params (group.id=6e50e4b8168340a8bc57713559ff23ae, role.id=8692e99e9463459f9b7caa96b3051b93, scope.project.id=f8a33963b3504bf196b09f575de6e329) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:14.310671 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd9543f-d5de-4702-b81a-e6abdc932fd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:14.311905 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd9543f-d5de-4702-b81a-e6abdc932fd8 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:14.320116 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 179/369] 10.222.0.71 () {56 vars in 1376 bytes} [Mon Jul 4 09:06:14 2022] GET /identity/v3/role_assignments?group.id=6e50e4b8168340a8bc57713559ff23ae&scope.project.id=f8a33963b3504bf196b09f575de6e329&role.id=8692e99e9463459f9b7caa96b3051b93 => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:14.321069 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd9543f-d5de-4702-b81a-e6abdc932fd8 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:14.331216 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952c162e-c930-42c1-949d-8ffbd4559a20 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:14.375911 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4204ea97-5a15-4f07-899a-c153bf64c398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:14.376974 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4204ea97-5a15-4f07-899a-c153bf64c398 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:14.377489 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4204ea97-5a15-4f07-899a-c153bf64c398 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:14.377968 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4204ea97-5a15-4f07-899a-c153bf64c398 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:14.378932 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4204ea97-5a15-4f07-899a-c153bf64c398 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:14.379933 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4204ea97-5a15-4f07-899a-c153bf64c398 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:14.381599 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4204ea97-5a15-4f07-899a-c153bf64c398 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:14.392113 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 191/370] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:06:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:14.393348 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4204ea97-5a15-4f07-899a-c153bf64c398 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:14.523047 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4204ea97-5a15-4f07-899a-c153bf64c398 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:14.564224 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c70dee-b450-4d6f-8041-860b6f783533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:14.565533 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c70dee-b450-4d6f-8041-860b6f783533 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:14.566021 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c70dee-b450-4d6f-8041-860b6f783533 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:14.566533 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c70dee-b450-4d6f-8041-860b6f783533 None admin] PATH_INFO: `/v3/users/384f8617e103485aa773cc65f1d4ca1e` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:14.568293 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c70dee-b450-4d6f-8041-860b6f783533 None admin] RBAC: Authorizing `identity:get_user(user_id=384f8617e103485aa773cc65f1d4ca1e)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:14.570388 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c70dee-b450-4d6f-8041-860b6f783533 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:14.573996 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 192/371] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 4 09:06:14 2022] GET /identity/v3/users/384f8617e103485aa773cc65f1d4ca1e => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:14.575061 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c70dee-b450-4d6f-8041-860b6f783533 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:14.585522 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd9543f-d5de-4702-b81a-e6abdc932fd8 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:14.591702 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:14.655398 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a3a6f-7f1c-40da-9003-30d4f092b7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:14.656070 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a3a6f-7f1c-40da-9003-30d4f092b7c7 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:14.656412 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a3a6f-7f1c-40da-9003-30d4f092b7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:14.656747 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a3a6f-7f1c-40da-9003-30d4f092b7c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:14.689565 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:14.706232 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c70dee-b450-4d6f-8041-860b6f783533 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:14.744706 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84a3a6f-7f1c-40da-9003-30d4f092b7c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:14.746235 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84a3a6f-7f1c-40da-9003-30d4f092b7c7 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:14.752769 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e93e0fc-8b42-4f69-956e-b8c679271603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:14.753733 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e93e0fc-8b42-4f69-956e-b8c679271603 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:14.754239 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e93e0fc-8b42-4f69-956e-b8c679271603 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:14.754736 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e93e0fc-8b42-4f69-956e-b8c679271603 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:14.835372 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e93e0fc-8b42-4f69-956e-b8c679271603 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:14.837658 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e93e0fc-8b42-4f69-956e-b8c679271603 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:14.851223 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:14.958337 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 193/372] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:06:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:06:14.959862 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e93e0fc-8b42-4f69-956e-b8c679271603 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:14.983151 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 180/373] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:06:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 662 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:06:14.985314 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a3a6f-7f1c-40da-9003-30d4f092b7c7 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:15.081653 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e93e0fc-8b42-4f69-956e-b8c679271603 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:15.095040 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:15.101727 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:15.123082 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c34f17-e567-4f87-a9dd-0e5aa9706181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:15.124123 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c34f17-e567-4f87-a9dd-0e5aa9706181 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:15.124616 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c34f17-e567-4f87-a9dd-0e5aa9706181 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:15.125122 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c34f17-e567-4f87-a9dd-0e5aa9706181 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:15.126155 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c34f17-e567-4f87-a9dd-0e5aa9706181 None admin] RBAC: Adding query filter params (role.id=56205a5df6514298a69fde673550f0aa, user.id=384f8617e103485aa773cc65f1d4ca1e, scope.system=all) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:15.126688 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c34f17-e567-4f87-a9dd-0e5aa9706181 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:15.128327 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c34f17-e567-4f87-a9dd-0e5aa9706181 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:15.142980 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 194/374] 10.222.0.71 () {56 vars in 1308 bytes} [Mon Jul 4 09:06:14 2022] GET /identity/v3/role_assignments?user.id=384f8617e103485aa773cc65f1d4ca1e&scope.system=all&role.id=56205a5df6514298a69fde673550f0aa => generated 510 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:15.144459 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c34f17-e567-4f87-a9dd-0e5aa9706181 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:15.258254 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a3a6f-7f1c-40da-9003-30d4f092b7c7 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:15.264063 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c34f17-e567-4f87-a9dd-0e5aa9706181 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:15.301696 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bee4deb-b23d-4cb4-bc30-033e600a75b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:15.303258 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee4deb-b23d-4cb4-bc30-033e600a75b0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:15.304022 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee4deb-b23d-4cb4-bc30-033e600a75b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:15.304517 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee4deb-b23d-4cb4-bc30-033e600a75b0 None admin] PATH_INFO: `/v3/projects/f8a33963b3504bf196b09f575de6e329` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:15.306220 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bee4deb-b23d-4cb4-bc30-033e600a75b0 None admin] RBAC: Authorizing `identity:get_project(project_id=f8a33963b3504bf196b09f575de6e329)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:15.307992 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bee4deb-b23d-4cb4-bc30-033e600a75b0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:15.311022 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 195/375] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 4 09:06:15 2022] GET /identity/v3/projects/f8a33963b3504bf196b09f575de6e329 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:15.316352 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bee4deb-b23d-4cb4-bc30-033e600a75b0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:15.346625 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cb4fbd-2a56-4176-aad0-466b2d1531c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:15.347448 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb4fbd-2a56-4176-aad0-466b2d1531c2 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:15.347734 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb4fbd-2a56-4176-aad0-466b2d1531c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:15.348016 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb4fbd-2a56-4176-aad0-466b2d1531c2 None admin] PATH_INFO: `/v3/projects/a34f9b2c1cbf4bceb4067648553a5416` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:15.349445 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cb4fbd-2a56-4176-aad0-466b2d1531c2 None admin] RBAC: Authorizing `identity:get_project(project_id=a34f9b2c1cbf4bceb4067648553a5416)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:15.350802 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cb4fbd-2a56-4176-aad0-466b2d1531c2 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:15.353122 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 181/376] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 4 09:06:14 2022] GET /identity/v3/projects/a34f9b2c1cbf4bceb4067648553a5416 => generated 344 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:15.358330 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cb4fbd-2a56-4176-aad0-466b2d1531c2 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:15.414136 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:15.506747 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bee4deb-b23d-4cb4-bc30-033e600a75b0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:15.553641 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de68e30a-47cb-49d0-bfd8-c324f91c75c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:15.554865 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de68e30a-47cb-49d0-bfd8-c324f91c75c9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:15.555298 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de68e30a-47cb-49d0-bfd8-c324f91c75c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:15.555655 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de68e30a-47cb-49d0-bfd8-c324f91c75c9 None admin] PATH_INFO: `/v3/groups/6e50e4b8168340a8bc57713559ff23ae` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:15.557632 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de68e30a-47cb-49d0-bfd8-c324f91c75c9 None admin] RBAC: Authorizing `identity:get_group(group_id=6e50e4b8168340a8bc57713559ff23ae)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:15.559094 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de68e30a-47cb-49d0-bfd8-c324f91c75c9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:15.561094 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 196/377] 10.222.0.71 () {56 vars in 1180 bytes} [Mon Jul 4 09:06:15 2022] GET /identity/v3/groups/6e50e4b8168340a8bc57713559ff23ae => generated 226 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:15.566753 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de68e30a-47cb-49d0-bfd8-c324f91c75c9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:15.577029 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cb4fbd-2a56-4176-aad0-466b2d1531c2 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:15.601281 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:15.640670 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59753645-7588-4f73-b59c-102a8e7abc75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:15.641574 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59753645-7588-4f73-b59c-102a8e7abc75 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:15.641864 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59753645-7588-4f73-b59c-102a8e7abc75 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:15.642143 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59753645-7588-4f73-b59c-102a8e7abc75 None admin] PATH_INFO: `/v3/groups/7c24c12b114748848903409974efea20` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:15.651600 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59753645-7588-4f73-b59c-102a8e7abc75 None admin] RBAC: Authorizing `identity:get_group(group_id=7c24c12b114748848903409974efea20)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:15.653092 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59753645-7588-4f73-b59c-102a8e7abc75 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:15.654810 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 182/378] 10.222.0.71 () {56 vars in 1180 bytes} [Mon Jul 4 09:06:15 2022] GET /identity/v3/groups/7c24c12b114748848903409974efea20 => generated 229 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:06:15.661489 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59753645-7588-4f73-b59c-102a8e7abc75 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:15.732214 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de68e30a-47cb-49d0-bfd8-c324f91c75c9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:15.735627 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:15.778098 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53f9328-d19b-4056-aefa-9fa817dbb572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:15.778767 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f9328-d19b-4056-aefa-9fa817dbb572 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:15.779214 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f9328-d19b-4056-aefa-9fa817dbb572 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:15.779498 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f9328-d19b-4056-aefa-9fa817dbb572 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:15.780768 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53f9328-d19b-4056-aefa-9fa817dbb572 None admin] RBAC: Adding query filter params (group.id=6e50e4b8168340a8bc57713559ff23ae, role.id=5fb58444d5604bf689014e27dccacb45, scope.project.id=f8a33963b3504bf196b09f575de6e329) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:15.781256 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53f9328-d19b-4056-aefa-9fa817dbb572 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:15.782713 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53f9328-d19b-4056-aefa-9fa817dbb572 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:15.791454 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 197/379] 10.222.0.71 () {56 vars in 1376 bytes} [Mon Jul 4 09:06:15 2022] GET /identity/v3/role_assignments?group.id=6e50e4b8168340a8bc57713559ff23ae&scope.project.id=f8a33963b3504bf196b09f575de6e329&role.id=5fb58444d5604bf689014e27dccacb45 => generated 260 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:15.792865 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398aca63-3540-4aba-9089-136c68cfd1ba None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:15.793190 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398aca63-3540-4aba-9089-136c68cfd1ba None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:15.793473 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398aca63-3540-4aba-9089-136c68cfd1ba None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:15.794843 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 198/380] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:15.798425 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59753645-7588-4f73-b59c-102a8e7abc75 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:15.801940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd626f-2026-4bfa-bff0-ebb294845eee None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:15.802203 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd626f-2026-4bfa-bff0-ebb294845eee None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:15.802487 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd626f-2026-4bfa-bff0-ebb294845eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:15.822750 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-74fd626f-2026-4bfa-bff0-ebb294845eee None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:15.824258 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-74fd626f-2026-4bfa-bff0-ebb294845eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:15.881847 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93960329-9ea9-4b38-b0dd-7ff854a8709d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:15.882493 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93960329-9ea9-4b38-b0dd-7ff854a8709d None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:15.882730 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93960329-9ea9-4b38-b0dd-7ff854a8709d None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:15.883070 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93960329-9ea9-4b38-b0dd-7ff854a8709d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:15.884002 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93960329-9ea9-4b38-b0dd-7ff854a8709d None admin] RBAC: Adding query filter params (group.id=7c24c12b114748848903409974efea20, role.id=f36e053b442540f08041ac7878760f65, scope.project.id=a34f9b2c1cbf4bceb4067648553a5416) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:15.884327 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93960329-9ea9-4b38-b0dd-7ff854a8709d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:15.885998 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93960329-9ea9-4b38-b0dd-7ff854a8709d None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:15.901528 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 183/381] 10.222.0.71 () {56 vars in 1376 bytes} [Mon Jul 4 09:06:15 2022] GET /identity/v3/role_assignments?group.id=7c24c12b114748848903409974efea20&scope.project.id=a34f9b2c1cbf4bceb4067648553a5416&role.id=f36e053b442540f08041ac7878760f65 => generated 260 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:15.909543 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 199/382] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:06:15.933613 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93960329-9ea9-4b38-b0dd-7ff854a8709d None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:16.016972 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:16.094303 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93960329-9ea9-4b38-b0dd-7ff854a8709d None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:16.136557 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1500eb69-3837-4137-acb6-0324fd18d702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:16.137318 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1500eb69-3837-4137-acb6-0324fd18d702 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:16.137648 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1500eb69-3837-4137-acb6-0324fd18d702 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:16.137888 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1500eb69-3837-4137-acb6-0324fd18d702 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:16.150388 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1500eb69-3837-4137-acb6-0324fd18d702 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:06:16.158177 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-1500eb69-3837-4137-acb6-0324fd18d702 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:06:16.158177 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:16.158177 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:16.158177 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:16.158177 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:16.158177 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:16.158177 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:16.158177 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:16.158177 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:16.158177 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:16.158177 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:16.158177 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:16.158177 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:06:16.158177 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:06:16.158177 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:06:16.158177 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:06:16.160264 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:06:16.160264 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:06:16.160264 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:06:16.160264 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:06:16.160264 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:16.160264 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:16.160264 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:16.160264 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:16.160264 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:16.160264 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:16.160264 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:16.160264 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:16.160264 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:16.160264 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:16.160264 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:16.160264 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:16.162248 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:16.162248 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:16.162248 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:16.162248 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:16.162248 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:06:16.162248 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:06:16.162248 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:06:16.162248 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:06:16.162248 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:06:16.162248 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:06:16.162248 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:06:16.162248 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:16.162248 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 184/383] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:06:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:16.166199 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fd626f-2026-4bfa-bff0-ebb294845eee None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:16.303310 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:16.308516 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fd626f-2026-4bfa-bff0-ebb294845eee None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:16.344019 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c91832-8e3f-4350-a53a-5935225d5309 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:16.344325 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c91832-8e3f-4350-a53a-5935225d5309 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:16.344584 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c91832-8e3f-4350-a53a-5935225d5309 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:16.347857 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 185/384] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:16.351730 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563eebaa-09d1-4e89-bd53-ceac80f732b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:16.351958 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33d0685-8568-42db-a833-9e911d0b7ebf None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:16.352114 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33d0685-8568-42db-a833-9e911d0b7ebf None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:16.352295 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33d0685-8568-42db-a833-9e911d0b7ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:16.352588 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563eebaa-09d1-4e89-bd53-ceac80f732b2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:16.353390 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563eebaa-09d1-4e89-bd53-ceac80f732b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:16.353676 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563eebaa-09d1-4e89-bd53-ceac80f732b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:16.354443 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563eebaa-09d1-4e89-bd53-ceac80f732b2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:16.354755 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563eebaa-09d1-4e89-bd53-ceac80f732b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:16.356203 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563eebaa-09d1-4e89-bd53-ceac80f732b2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:16.365977 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 200/385] 10.222.0.71 () {56 vars in 1101 bytes} [Mon Jul 4 09:06:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:16.371523 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-f33d0685-8568-42db-a833-9e911d0b7ebf None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:16.371738 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563eebaa-09d1-4e89-bd53-ceac80f732b2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:16.372843 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f33d0685-8568-42db-a833-9e911d0b7ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:16.461070 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 186/386] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:06:16.508504 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33d0685-8568-42db-a833-9e911d0b7ebf None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:16.528065 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563eebaa-09d1-4e89-bd53-ceac80f732b2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:16.562294 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:16.578793 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de0c5b4-bbcf-4521-ad63-cf2b77cfdfa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:16.580666 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de0c5b4-bbcf-4521-ad63-cf2b77cfdfa5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:16.581218 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de0c5b4-bbcf-4521-ad63-cf2b77cfdfa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:16.581718 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de0c5b4-bbcf-4521-ad63-cf2b77cfdfa5 None admin] PATH_INFO: `/v3/groups/6e50e4b8168340a8bc57713559ff23ae` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:16.583574 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de0c5b4-bbcf-4521-ad63-cf2b77cfdfa5 None admin] RBAC: Authorizing `identity:get_group(group_id=6e50e4b8168340a8bc57713559ff23ae)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:16.585622 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de0c5b4-bbcf-4521-ad63-cf2b77cfdfa5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:16.589163 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 201/387] 10.222.0.71 () {56 vars in 1180 bytes} [Mon Jul 4 09:06:16 2022] GET /identity/v3/groups/6e50e4b8168340a8bc57713559ff23ae => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:06:16.594662 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de0c5b4-bbcf-4521-ad63-cf2b77cfdfa5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:16.655958 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:16.693452 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33d0685-8568-42db-a833-9e911d0b7ebf None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:16.738389 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6de716-639b-44c2-969b-8624c86896bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:16.739595 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6de716-639b-44c2-969b-8624c86896bd None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:16.740114 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6de716-639b-44c2-969b-8624c86896bd None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:16.740592 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6de716-639b-44c2-969b-8624c86896bd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:16.755238 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6de716-639b-44c2-969b-8624c86896bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:06:16.764181 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-ae6de716-639b-44c2-969b-8624c86896bd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:06:16.764181 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:16.764181 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:16.764181 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:16.764181 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:16.764181 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:16.764181 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:16.764181 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:16.764181 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:16.764181 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:16.764181 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:16.764181 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:16.764181 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:06:16.764181 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:06:16.764181 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:06:16.764181 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:06:16.766340 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:06:16.766340 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:06:16.766340 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:06:16.766340 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:06:16.766340 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:16.766340 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:16.766340 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:16.766340 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:16.766340 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:16.766340 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:16.766340 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:16.766340 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:16.766340 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:16.766340 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:16.766340 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:16.766340 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:16.767160 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:16.767160 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:16.767160 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:16.767160 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:16.767160 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:06:16.767160 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:06:16.767160 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:06:16.767160 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:06:16.767160 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:06:16.767160 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:06:16.767160 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:06:16.767160 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:16.769754 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 187/388] 10.222.0.71 () {56 vars in 1114 bytes} [Mon Jul 4 09:06:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:06:16.778654 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6de716-639b-44c2-969b-8624c86896bd None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:16.803023 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de0c5b4-bbcf-4521-ad63-cf2b77cfdfa5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:16.893454 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6de716-639b-44c2-969b-8624c86896bd None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:16.907705 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:16.925766 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6310a36-1180-4d89-a448-484338ae9f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:16.926661 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6310a36-1180-4d89-a448-484338ae9f61 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:16.927044 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6310a36-1180-4d89-a448-484338ae9f61 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:16.927218 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6310a36-1180-4d89-a448-484338ae9f61 None admin] PATH_INFO: `/v3/projects/e192ac889bad428985088eea5ce750ab` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:16.932420 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6310a36-1180-4d89-a448-484338ae9f61 None admin] RBAC: Authorizing `identity:get_project(project_id=e192ac889bad428985088eea5ce750ab)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:16.934070 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6310a36-1180-4d89-a448-484338ae9f61 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:16.934893 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61044dff-3521-44bb-8828-e286b9f2d6b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:16.935789 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61044dff-3521-44bb-8828-e286b9f2d6b6 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:16.936593 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 202/389] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 4 09:06:16 2022] GET /identity/v3/projects/e192ac889bad428985088eea5ce750ab => generated 298 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:16.937211 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61044dff-3521-44bb-8828-e286b9f2d6b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:16.937730 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61044dff-3521-44bb-8828-e286b9f2d6b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:16.938739 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61044dff-3521-44bb-8828-e286b9f2d6b6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:16.939245 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61044dff-3521-44bb-8828-e286b9f2d6b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:16.940859 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61044dff-3521-44bb-8828-e286b9f2d6b6 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:16.942374 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6310a36-1180-4d89-a448-484338ae9f61 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:16.952285 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 188/390] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 4 09:06:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:16.968259 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61044dff-3521-44bb-8828-e286b9f2d6b6 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:17.088851 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61044dff-3521-44bb-8828-e286b9f2d6b6 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:17.128641 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f85b8ec-27a9-41b9-8fb7-05dbb8a72425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:17.129933 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85b8ec-27a9-41b9-8fb7-05dbb8a72425 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:17.130424 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85b8ec-27a9-41b9-8fb7-05dbb8a72425 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:17.130900 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85b8ec-27a9-41b9-8fb7-05dbb8a72425 None admin] PATH_INFO: `/v3/groups/6e50e4b8168340a8bc57713559ff23ae` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:17.132296 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f85b8ec-27a9-41b9-8fb7-05dbb8a72425 None admin] RBAC: Authorizing `identity:get_group(group_id=6e50e4b8168340a8bc57713559ff23ae)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:17.134891 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f85b8ec-27a9-41b9-8fb7-05dbb8a72425 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:17.137201 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 189/391] 10.222.0.71 () {56 vars in 1180 bytes} [Mon Jul 4 09:06:16 2022] GET /identity/v3/groups/6e50e4b8168340a8bc57713559ff23ae => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:06:17.144257 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f85b8ec-27a9-41b9-8fb7-05dbb8a72425 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:17.189191 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:17.189691 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:17.199537 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6310a36-1180-4d89-a448-484338ae9f61 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:17.248847 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2457242a-fe02-46e0-9d18-e563ba10a764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:17.248847 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457242a-fe02-46e0-9d18-e563ba10a764 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:17.248847 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457242a-fe02-46e0-9d18-e563ba10a764 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:17.248847 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457242a-fe02-46e0-9d18-e563ba10a764 None admin] PATH_INFO: `/v3/projects/e192ac889bad428985088eea5ce750ab/groups/6e50e4b8168340a8bc57713559ff23ae/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:17.251445 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2457242a-fe02-46e0-9d18-e563ba10a764 None admin] RBAC: Authorizing `identity:create_grant(project_id=e192ac889bad428985088eea5ce750ab, group_id=6e50e4b8168340a8bc57713559ff23ae, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:17.253067 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2457242a-fe02-46e0-9d18-e563ba10a764 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:17.270831 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 203/392] 10.222.0.71 () {58 vars in 1442 bytes} [Mon Jul 4 09:06:16 2022] PUT /identity/v3/projects/e192ac889bad428985088eea5ce750ab/groups/6e50e4b8168340a8bc57713559ff23ae/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:06:17.272425 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa7753-98c1-48a6-8723-a509f0afba1c None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:17.272724 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa7753-98c1-48a6-8723-a509f0afba1c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:17.273514 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa7753-98c1-48a6-8723-a509f0afba1c None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:17.274876 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 204/393] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:17.288831 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec8b03-1f5f-44e9-a408-ff07b03652cd None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:17.289158 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec8b03-1f5f-44e9-a408-ff07b03652cd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:17.289438 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec8b03-1f5f-44e9-a408-ff07b03652cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:17.300571 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f85b8ec-27a9-41b9-8fb7-05dbb8a72425 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:17.312947 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1dec8b03-1f5f-44e9-a408-ff07b03652cd None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:17.314409 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1dec8b03-1f5f-44e9-a408-ff07b03652cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:17.353815 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6312135-0e36-4d7b-afe4-49dbc2b40403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:17.354782 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6312135-0e36-4d7b-afe4-49dbc2b40403 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:17.355074 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6312135-0e36-4d7b-afe4-49dbc2b40403 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:17.355407 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6312135-0e36-4d7b-afe4-49dbc2b40403 None admin] PATH_INFO: `/v3/projects/e192ac889bad428985088eea5ce750ab` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:17.357385 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6312135-0e36-4d7b-afe4-49dbc2b40403 None admin] RBAC: Authorizing `identity:get_project(project_id=e192ac889bad428985088eea5ce750ab)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:17.358655 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6312135-0e36-4d7b-afe4-49dbc2b40403 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:17.365282 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 190/394] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 4 09:06:17 2022] GET /identity/v3/projects/e192ac889bad428985088eea5ce750ab => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:17.367429 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6312135-0e36-4d7b-afe4-49dbc2b40403 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:17.425771 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 205/395] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:06:17.453261 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dec8b03-1f5f-44e9-a408-ff07b03652cd None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:17.476708 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:17.487307 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6312135-0e36-4d7b-afe4-49dbc2b40403 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:17.527375 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7814305-8d7f-4d66-9a37-a61d60628035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:17.528294 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7814305-8d7f-4d66-9a37-a61d60628035 None admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:17.528578 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7814305-8d7f-4d66-9a37-a61d60628035 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:17.528861 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7814305-8d7f-4d66-9a37-a61d60628035 None admin] PATH_INFO: `/v3/projects/e192ac889bad428985088eea5ce750ab/groups/6e50e4b8168340a8bc57713559ff23ae/roles/5fb58444d5604bf689014e27dccacb45` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:17.531961 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7814305-8d7f-4d66-9a37-a61d60628035 None admin] RBAC: Authorizing `identity:create_grant(project_id=e192ac889bad428985088eea5ce750ab, group_id=6e50e4b8168340a8bc57713559ff23ae, role_id=5fb58444d5604bf689014e27dccacb45)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:17.533229 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7814305-8d7f-4d66-9a37-a61d60628035 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:17.555002 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 191/396] 10.222.0.71 () {58 vars in 1442 bytes} [Mon Jul 4 09:06:17 2022] PUT /identity/v3/projects/e192ac889bad428985088eea5ce750ab/groups/6e50e4b8168340a8bc57713559ff23ae/roles/5fb58444d5604bf689014e27dccacb45 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:06:17.560946 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:17.580717 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dec8b03-1f5f-44e9-a408-ff07b03652cd None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:17.622792 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bdf075-9786-4496-9655-0a51f0f4f60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:17.623627 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bdf075-9786-4496-9655-0a51f0f4f60b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:17.623916 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bdf075-9786-4496-9655-0a51f0f4f60b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:17.624254 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bdf075-9786-4496-9655-0a51f0f4f60b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:17.637254 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bdf075-9786-4496-9655-0a51f0f4f60b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:06:17.645621 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-a6bdf075-9786-4496-9655-0a51f0f4f60b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:06:17.645621 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:17.645621 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:17.645621 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:17.645621 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:17.645621 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:17.645621 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:17.645621 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:17.645621 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:17.645621 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:17.645621 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:17.645621 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:17.645621 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:06:17.645621 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:06:17.645621 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:06:17.645621 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:06:17.646584 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:06:17.646584 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:06:17.646584 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:06:17.646584 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:06:17.646584 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:17.646584 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:17.646584 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:17.646584 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:17.646584 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:17.646584 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:17.646584 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:17.646584 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:17.646584 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:17.646584 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:17.646584 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:17.646584 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:17.647965 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:17.647965 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:17.647965 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:17.647965 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:17.647965 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:06:17.647965 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:06:17.647965 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:06:17.647965 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:06:17.647965 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:06:17.647965 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:06:17.647965 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:06:17.647965 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:17.647965 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 206/397] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:06:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:17.653873 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7814305-8d7f-4d66-9a37-a61d60628035 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:17.792176 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7814305-8d7f-4d66-9a37-a61d60628035 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:17.835681 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d8c1e-384d-45b1-aa31-307ffb8a5f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:17.836365 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d8c1e-384d-45b1-aa31-307ffb8a5f47 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:17.836647 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d8c1e-384d-45b1-aa31-307ffb8a5f47 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:17.837355 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d8c1e-384d-45b1-aa31-307ffb8a5f47 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:17.838616 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8d8c1e-384d-45b1-aa31-307ffb8a5f47 None admin] RBAC: Adding query filter params (name=member) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:17.838931 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8d8c1e-384d-45b1-aa31-307ffb8a5f47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:17.840362 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8d8c1e-384d-45b1-aa31-307ffb8a5f47 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:17.849399 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 192/398] 10.222.0.71 () {56 vars in 1101 bytes} [Mon Jul 4 09:06:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:17.855827 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bdf075-9786-4496-9655-0a51f0f4f60b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:17.965736 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bdf075-9786-4496-9655-0a51f0f4f60b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:18.001748 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3536af0a-9938-421e-bd4e-213408b2ce0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:18.002608 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3536af0a-9938-421e-bd4e-213408b2ce0a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:18.002891 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3536af0a-9938-421e-bd4e-213408b2ce0a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:18.003165 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3536af0a-9938-421e-bd4e-213408b2ce0a None admin] PATH_INFO: `/v3/groups/6e50e4b8168340a8bc57713559ff23ae` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:18.004326 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3536af0a-9938-421e-bd4e-213408b2ce0a None admin] RBAC: Authorizing `identity:get_group(group_id=6e50e4b8168340a8bc57713559ff23ae)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:18.005777 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3536af0a-9938-421e-bd4e-213408b2ce0a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:18.007673 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 207/399] 10.222.0.71 () {56 vars in 1180 bytes} [Mon Jul 4 09:06:17 2022] GET /identity/v3/groups/6e50e4b8168340a8bc57713559ff23ae => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:18.014477 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d8c1e-384d-45b1-aa31-307ffb8a5f47 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:18.031020 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:18.137378 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d8c1e-384d-45b1-aa31-307ffb8a5f47 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:18.174309 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ef5f6-e673-4255-9d3a-20f6727e4174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:18.175178 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ef5f6-e673-4255-9d3a-20f6727e4174 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:18.175462 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ef5f6-e673-4255-9d3a-20f6727e4174 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:18.175742 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ef5f6-e673-4255-9d3a-20f6727e4174 None admin] PATH_INFO: `/v3/projects/f8a33963b3504bf196b09f575de6e329` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:18.177016 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36ef5f6-e673-4255-9d3a-20f6727e4174 None admin] RBAC: Authorizing `identity:get_project(project_id=f8a33963b3504bf196b09f575de6e329)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:18.178478 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36ef5f6-e673-4255-9d3a-20f6727e4174 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:18.181751 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 193/400] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 4 09:06:18 2022] GET /identity/v3/projects/f8a33963b3504bf196b09f575de6e329 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:18.187499 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3536af0a-9938-421e-bd4e-213408b2ce0a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:18.205738 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:18.296616 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3536af0a-9938-421e-bd4e-213408b2ce0a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:18.333246 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31d5868-e285-4806-a519-823b9d410f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:18.334124 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31d5868-e285-4806-a519-823b9d410f41 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:18.334405 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31d5868-e285-4806-a519-823b9d410f41 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:18.334683 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31d5868-e285-4806-a519-823b9d410f41 None admin] PATH_INFO: `/v3/projects/f8a33963b3504bf196b09f575de6e329/groups/6e50e4b8168340a8bc57713559ff23ae/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:18.336649 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31d5868-e285-4806-a519-823b9d410f41 None admin] RBAC: Authorizing `identity:create_grant(project_id=f8a33963b3504bf196b09f575de6e329, group_id=6e50e4b8168340a8bc57713559ff23ae, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:18.339652 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31d5868-e285-4806-a519-823b9d410f41 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:18.359209 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 208/401] 10.222.0.71 () {58 vars in 1442 bytes} [Mon Jul 4 09:06:18 2022] PUT /identity/v3/projects/f8a33963b3504bf196b09f575de6e329/groups/6e50e4b8168340a8bc57713559ff23ae/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:06:18.550572 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a3aaef-e723-4090-9cd1-6731d0030df0 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:18.550941 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a3aaef-e723-4090-9cd1-6731d0030df0 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:18.551153 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a3aaef-e723-4090-9cd1-6731d0030df0 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:18.552654 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 194/402] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:18.558429 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91062c61-f48c-4e7e-ba08-d558b64963bb None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:18.558718 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91062c61-f48c-4e7e-ba08-d558b64963bb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:18.559002 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91062c61-f48c-4e7e-ba08-d558b64963bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:18.577986 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-91062c61-f48c-4e7e-ba08-d558b64963bb None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:18.579267 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-91062c61-f48c-4e7e-ba08-d558b64963bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:18.657822 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 209/403] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:06:18.690836 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a3aaef-e723-4090-9cd1-6731d0030df0 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:18.779199 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:18.800679 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a3aaef-e723-4090-9cd1-6731d0030df0 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:18.837825 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cac02e4-a86d-48ba-8672-d934abb15df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:18.839058 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac02e4-a86d-48ba-8672-d934abb15df1 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:18.839542 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac02e4-a86d-48ba-8672-d934abb15df1 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:18.840015 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac02e4-a86d-48ba-8672-d934abb15df1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:18.850948 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cac02e4-a86d-48ba-8672-d934abb15df1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:06:18.859706 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-6cac02e4-a86d-48ba-8672-d934abb15df1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:06:18.859706 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:18.859706 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:18.859706 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:18.859706 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:18.859706 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:18.859706 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:18.859706 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:18.859706 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:18.859706 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:18.859706 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:18.859706 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:18.859706 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:06:18.859706 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:06:18.859706 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:06:18.859706 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:06:18.860623 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:06:18.860623 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:06:18.860623 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:06:18.860623 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:06:18.860623 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:18.860623 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:18.860623 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:18.860623 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:18.860623 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:18.860623 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:18.860623 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:18.860623 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:18.860623 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:18.860623 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:18.860623 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:18.860623 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:18.861448 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:18.861448 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:18.861448 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:18.861448 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:18.861448 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:06:18.861448 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:06:18.861448 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:06:18.861448 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:06:18.861448 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:06:18.861448 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:06:18.861448 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:06:18.861448 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:18.868995 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 195/404] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 4 09:06:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:18.870647 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91062c61-f48c-4e7e-ba08-d558b64963bb None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:18.873866 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:18.963593 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf842a-b7fa-4831-814e-49f27c1a43b0 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:18.964235 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf842a-b7fa-4831-814e-49f27c1a43b0 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:18.964619 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf842a-b7fa-4831-814e-49f27c1a43b0 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:18.966129 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 196/405] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:06:18.971126 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c9c164-8991-4db6-8a78-6fc7bc13ed13 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:18.971541 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c9c164-8991-4db6-8a78-6fc7bc13ed13 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:18.971920 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c9c164-8991-4db6-8a78-6fc7bc13ed13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:18.998466 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-04c9c164-8991-4db6-8a78-6fc7bc13ed13 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:18.999771 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-04c9c164-8991-4db6-8a78-6fc7bc13ed13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:19.023143 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91062c61-f48c-4e7e-ba08-d558b64963bb None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:19.090816 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52c539f-363e-4ae0-a609-576e6c035f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:19.091499 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c539f-363e-4ae0-a609-576e6c035f9d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:19.091784 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c539f-363e-4ae0-a609-576e6c035f9d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:19.092063 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c539f-363e-4ae0-a609-576e6c035f9d None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:19.092818 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52c539f-363e-4ae0-a609-576e6c035f9d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:19.093148 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52c539f-363e-4ae0-a609-576e6c035f9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:19.094519 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52c539f-363e-4ae0-a609-576e6c035f9d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:19.113344 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 210/406] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:06:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:19.114913 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52c539f-363e-4ae0-a609-576e6c035f9d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:19.144214 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 197/407] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:06:19.202719 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c9c164-8991-4db6-8a78-6fc7bc13ed13 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:19.221268 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:19.273380 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52c539f-363e-4ae0-a609-576e6c035f9d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:19.320752 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a9043f-1f28-4568-9ebf-c7975547796e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:19.321758 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a9043f-1f28-4568-9ebf-c7975547796e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:19.322040 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a9043f-1f28-4568-9ebf-c7975547796e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:19.322318 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a9043f-1f28-4568-9ebf-c7975547796e None admin] PATH_INFO: `/v3/groups/7c24c12b114748848903409974efea20` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:19.323505 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a9043f-1f28-4568-9ebf-c7975547796e None admin] RBAC: Authorizing `identity:get_group(group_id=7c24c12b114748848903409974efea20)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:19.325190 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a9043f-1f28-4568-9ebf-c7975547796e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:19.331639 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 211/408] 10.222.0.71 () {56 vars in 1180 bytes} [Mon Jul 4 09:06:19 2022] GET /identity/v3/groups/7c24c12b114748848903409974efea20 => generated 229 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:06:19.332841 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a9043f-1f28-4568-9ebf-c7975547796e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:19.341988 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:19.379774 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c9c164-8991-4db6-8a78-6fc7bc13ed13 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:19.418272 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bca9e70-3334-41b4-b53f-004163f9589d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:19.419096 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bca9e70-3334-41b4-b53f-004163f9589d None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:19.419385 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bca9e70-3334-41b4-b53f-004163f9589d None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:19.419668 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bca9e70-3334-41b4-b53f-004163f9589d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:19.431160 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bca9e70-3334-41b4-b53f-004163f9589d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:06:19.439827 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-3bca9e70-3334-41b4-b53f-004163f9589d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:06:19.439827 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:19.439827 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:19.439827 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:19.439827 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:19.439827 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:19.439827 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:19.439827 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:19.439827 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:19.439827 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:19.439827 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:19.439827 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:19.439827 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:06:19.439827 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:06:19.439827 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:06:19.439827 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:06:19.440743 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:06:19.440743 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:06:19.440743 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:06:19.440743 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:06:19.440743 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:19.440743 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:19.440743 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:19.440743 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:19.440743 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:19.440743 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:19.440743 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:19.440743 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:19.440743 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:19.440743 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:19.440743 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:19.440743 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:19.441626 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:19.441626 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:19.441626 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:19.441626 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:19.441626 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:06:19.441626 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:06:19.441626 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:06:19.441626 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:06:19.441626 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:06:19.441626 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:06:19.441626 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:06:19.441626 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:19.441626 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 198/409] 10.222.0.71 () {56 vars in 1114 bytes} [Mon Jul 4 09:06:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:06:19.447228 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bca9e70-3334-41b4-b53f-004163f9589d None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:19.555522 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a9043f-1f28-4568-9ebf-c7975547796e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:19.570631 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bca9e70-3334-41b4-b53f-004163f9589d None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:19.601202 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:19.614790 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22555564-e463-4ec3-b416-76b4cf98c8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:19.615663 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22555564-e463-4ec3-b416-76b4cf98c8eb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:19.615949 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22555564-e463-4ec3-b416-76b4cf98c8eb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:19.616232 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22555564-e463-4ec3-b416-76b4cf98c8eb None admin] PATH_INFO: `/v3/projects/a34f9b2c1cbf4bceb4067648553a5416` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:19.617472 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22555564-e463-4ec3-b416-76b4cf98c8eb None admin] RBAC: Authorizing `identity:get_project(project_id=a34f9b2c1cbf4bceb4067648553a5416)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:19.619063 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22555564-e463-4ec3-b416-76b4cf98c8eb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:19.620781 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 212/410] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 4 09:06:19 2022] GET /identity/v3/projects/a34f9b2c1cbf4bceb4067648553a5416 => generated 344 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:06:19.622852 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360eca12-f57a-4c8e-aa27-66426ef42a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:19.623545 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360eca12-f57a-4c8e-aa27-66426ef42a88 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:19.623837 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360eca12-f57a-4c8e-aa27-66426ef42a88 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:19.624126 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360eca12-f57a-4c8e-aa27-66426ef42a88 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:19.624890 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360eca12-f57a-4c8e-aa27-66426ef42a88 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:19.625307 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360eca12-f57a-4c8e-aa27-66426ef42a88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:19.626714 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360eca12-f57a-4c8e-aa27-66426ef42a88 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:19.626890 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22555564-e463-4ec3-b416-76b4cf98c8eb None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:19.636735 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:19.647249 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 199/411] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 4 09:06:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:19.653678 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360eca12-f57a-4c8e-aa27-66426ef42a88 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:19.806296 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360eca12-f57a-4c8e-aa27-66426ef42a88 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:19.810225 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22555564-e463-4ec3-b416-76b4cf98c8eb None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:19.843745 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8713d7f-5004-4ad9-a1e0-ee43ea973b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:19.844967 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8713d7f-5004-4ad9-a1e0-ee43ea973b8b None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:19.845758 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8713d7f-5004-4ad9-a1e0-ee43ea973b8b None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:19.846450 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8713d7f-5004-4ad9-a1e0-ee43ea973b8b None admin] PATH_INFO: `/v3/groups/6e50e4b8168340a8bc57713559ff23ae` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:19.849868 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8713d7f-5004-4ad9-a1e0-ee43ea973b8b None admin] RBAC: Authorizing `identity:get_group(group_id=6e50e4b8168340a8bc57713559ff23ae)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:19.851515 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8713d7f-5004-4ad9-a1e0-ee43ea973b8b None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:19.854049 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 200/412] 10.222.0.71 () {56 vars in 1180 bytes} [Mon Jul 4 09:06:19 2022] GET /identity/v3/groups/6e50e4b8168340a8bc57713559ff23ae => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:19.864030 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8713d7f-5004-4ad9-a1e0-ee43ea973b8b None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:19.892763 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202d7b81-db1f-47a5-b446-54afe8a36be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:19.895174 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d7b81-db1f-47a5-b446-54afe8a36be1 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:19.896614 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d7b81-db1f-47a5-b446-54afe8a36be1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:19.897104 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d7b81-db1f-47a5-b446-54afe8a36be1 None admin] PATH_INFO: `/v3/projects/a34f9b2c1cbf4bceb4067648553a5416/groups/7c24c12b114748848903409974efea20/roles/f36e053b442540f08041ac7878760f65` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:19.900166 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202d7b81-db1f-47a5-b446-54afe8a36be1 None admin] RBAC: Authorizing `identity:create_grant(project_id=a34f9b2c1cbf4bceb4067648553a5416, group_id=7c24c12b114748848903409974efea20, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:19.902478 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202d7b81-db1f-47a5-b446-54afe8a36be1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:19.926087 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 213/413] 10.222.0.71 () {58 vars in 1442 bytes} [Mon Jul 4 09:06:19 2022] PUT /identity/v3/projects/a34f9b2c1cbf4bceb4067648553a5416/groups/7c24c12b114748848903409974efea20/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:06:20.005006 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:20.080501 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8713d7f-5004-4ad9-a1e0-ee43ea973b8b None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:20.124479 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fa0a0e-d7be-415b-a145-c451cc3b6fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:20.125379 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fa0a0e-d7be-415b-a145-c451cc3b6fb7 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:20.125664 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fa0a0e-d7be-415b-a145-c451cc3b6fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:20.125942 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fa0a0e-d7be-415b-a145-c451cc3b6fb7 None admin] PATH_INFO: `/v3/projects/f8a33963b3504bf196b09f575de6e329` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:20.127141 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fa0a0e-d7be-415b-a145-c451cc3b6fb7 None admin] RBAC: Authorizing `identity:get_project(project_id=f8a33963b3504bf196b09f575de6e329)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:20.128584 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fa0a0e-d7be-415b-a145-c451cc3b6fb7 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:20.130617 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 201/414] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 4 09:06:19 2022] GET /identity/v3/projects/f8a33963b3504bf196b09f575de6e329 => generated 302 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:06:20.137009 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202d7b81-db1f-47a5-b446-54afe8a36be1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:20.157350 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4c83c-ca76-4bdd-bdef-42cd52184bae None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:20.158240 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4c83c-ca76-4bdd-bdef-42cd52184bae None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:20.158738 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4c83c-ca76-4bdd-bdef-42cd52184bae None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:20.160332 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 202/415] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:06:20.167581 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28f9dc-3fea-47cc-ac5c-041c7999061e None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:20.168127 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28f9dc-3fea-47cc-ac5c-041c7999061e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:20.168617 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28f9dc-3fea-47cc-ac5c-041c7999061e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:20.201295 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-bd28f9dc-3fea-47cc-ac5c-041c7999061e None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:20.202861 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-bd28f9dc-3fea-47cc-ac5c-041c7999061e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:20.214144 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:20.276128 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202d7b81-db1f-47a5-b446-54afe8a36be1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:20.313300 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 203/416] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:06:20.338918 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22148c1-5475-46a8-8f1f-2811df82de19 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:20.339200 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22148c1-5475-46a8-8f1f-2811df82de19 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:20.339407 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22148c1-5475-46a8-8f1f-2811df82de19 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:20.341058 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 204/417] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:06:20.346143 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ba465-2175-4887-b06f-5ccaf9dda612 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:20.346437 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ba465-2175-4887-b06f-5ccaf9dda612 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:20.346724 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ba465-2175-4887-b06f-5ccaf9dda612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:20.359952 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa2325f-f390-40ab-957d-c77ea00c9bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:20.360846 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa2325f-f390-40ab-957d-c77ea00c9bdf None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:20.361173 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa2325f-f390-40ab-957d-c77ea00c9bdf None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:20.361457 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa2325f-f390-40ab-957d-c77ea00c9bdf None admin] PATH_INFO: `/v3/projects/f8a33963b3504bf196b09f575de6e329/groups/6e50e4b8168340a8bc57713559ff23ae/roles/5fb58444d5604bf689014e27dccacb45` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:20.363728 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa2325f-f390-40ab-957d-c77ea00c9bdf None admin] RBAC: Authorizing `identity:create_grant(project_id=f8a33963b3504bf196b09f575de6e329, group_id=6e50e4b8168340a8bc57713559ff23ae, role_id=5fb58444d5604bf689014e27dccacb45)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:20.364304 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-010ba465-2175-4887-b06f-5ccaf9dda612 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:20.365223 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa2325f-f390-40ab-957d-c77ea00c9bdf None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:20.365524 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-010ba465-2175-4887-b06f-5ccaf9dda612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:20.388264 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 214/418] 10.222.0.71 () {58 vars in 1442 bytes} [Mon Jul 4 09:06:20 2022] PUT /identity/v3/projects/f8a33963b3504bf196b09f575de6e329/groups/6e50e4b8168340a8bc57713559ff23ae/roles/5fb58444d5604bf689014e27dccacb45 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:06:20.389594 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa2325f-f390-40ab-957d-c77ea00c9bdf None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:20.516954 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:20.518631 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa2325f-f390-40ab-957d-c77ea00c9bdf None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:20.548378 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:20.549563 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 205/419] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:06:20.562123 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0035c0eb-8dd2-4dd9-a38d-9a64ce5a3a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:20.563049 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0035c0eb-8dd2-4dd9-a38d-9a64ce5a3a94 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:20.563341 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0035c0eb-8dd2-4dd9-a38d-9a64ce5a3a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:20.563624 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0035c0eb-8dd2-4dd9-a38d-9a64ce5a3a94 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:20.575567 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0035c0eb-8dd2-4dd9-a38d-9a64ce5a3a94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:06:20.583638 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-0035c0eb-8dd2-4dd9-a38d-9a64ce5a3a94 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:06:20.583638 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:20.583638 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:20.583638 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:20.583638 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:20.583638 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:20.583638 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:20.583638 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:20.583638 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:20.583638 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:20.583638 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:20.583638 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:20.583638 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:06:20.583638 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:06:20.583638 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:06:20.583638 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:06:20.584583 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:06:20.584583 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:06:20.584583 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:06:20.584583 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:06:20.584583 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:20.584583 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:20.584583 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:20.584583 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:20.584583 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:20.584583 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:20.584583 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:20.584583 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:20.584583 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:20.584583 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:20.584583 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:20.584583 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:20.585429 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:20.585429 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:20.585429 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:20.585429 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:20.585429 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:06:20.585429 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:06:20.585429 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:06:20.585429 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:06:20.585429 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:06:20.585429 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:06:20.585429 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:06:20.585429 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:20.585429 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 215/420] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:06:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:20.591820 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010ba465-2175-4887-b06f-5ccaf9dda612 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:20.592691 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0035c0eb-8dd2-4dd9-a38d-9a64ce5a3a94 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:20.719958 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010ba465-2175-4887-b06f-5ccaf9dda612 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:20.725903 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0035c0eb-8dd2-4dd9-a38d-9a64ce5a3a94 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:20.758869 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc73242-8249-4b06-bf4a-280f2ad768d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:20.759865 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc73242-8249-4b06-bf4a-280f2ad768d9 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:20.760165 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc73242-8249-4b06-bf4a-280f2ad768d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:20.760451 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc73242-8249-4b06-bf4a-280f2ad768d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:20.761269 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc73242-8249-4b06-bf4a-280f2ad768d9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:20.761695 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc73242-8249-4b06-bf4a-280f2ad768d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:20.763645 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc73242-8249-4b06-bf4a-280f2ad768d9 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:20.772177 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfadc08-2fdd-4996-a8a6-0013a7df90d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:20.773360 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfadc08-2fdd-4996-a8a6-0013a7df90d5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:20.773817 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 206/421] 10.222.0.71 () {56 vars in 1101 bytes} [Mon Jul 4 09:06:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:20.774042 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfadc08-2fdd-4996-a8a6-0013a7df90d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:20.774870 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfadc08-2fdd-4996-a8a6-0013a7df90d5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:20.786028 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc73242-8249-4b06-bf4a-280f2ad768d9 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:20.793405 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfadc08-2fdd-4996-a8a6-0013a7df90d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:06:20.805061 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-ccfadc08-2fdd-4996-a8a6-0013a7df90d5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:06:20.805061 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:20.805061 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:20.805061 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:20.805061 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:20.805061 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:20.805061 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:20.805061 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:20.805061 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:20.805061 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:20.805061 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:20.805061 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:20.805061 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:06:20.805061 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:06:20.805061 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:06:20.805061 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:06:20.806637 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:06:20.806637 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:06:20.806637 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:06:20.806637 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:06:20.806637 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:20.806637 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:20.806637 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:20.806637 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:20.806637 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:20.806637 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:20.806637 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:20.806637 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:20.806637 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:20.806637 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:20.806637 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:20.806637 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:20.807459 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:20.807459 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:20.807459 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:20.807459 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:20.807459 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:06:20.807459 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:06:20.807459 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:06:20.807459 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:06:20.807459 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:06:20.807459 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:06:20.807459 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:06:20.807459 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:20.807459 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 216/422] 10.222.0.71 () {56 vars in 1114 bytes} [Mon Jul 4 09:06:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:06:20.811703 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfadc08-2fdd-4996-a8a6-0013a7df90d5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:20.864502 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:20.930330 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:20.967582 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfadc08-2fdd-4996-a8a6-0013a7df90d5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:21.012066 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc73242-8249-4b06-bf4a-280f2ad768d9 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:21.028715 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee35c70-50ae-42f1-af44-475a2bac6ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:21.029572 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee35c70-50ae-42f1-af44-475a2bac6ba4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:21.029869 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee35c70-50ae-42f1-af44-475a2bac6ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:21.030186 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee35c70-50ae-42f1-af44-475a2bac6ba4 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:21.031049 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee35c70-50ae-42f1-af44-475a2bac6ba4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:21.031368 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee35c70-50ae-42f1-af44-475a2bac6ba4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:21.032778 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee35c70-50ae-42f1-af44-475a2bac6ba4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:21.043748 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 217/423] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 4 09:06:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:21.045070 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515bc15a-3cb7-4a8d-bebd-723b41dd5506 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:21.045429 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515bc15a-3cb7-4a8d-bebd-723b41dd5506 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:21.045722 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515bc15a-3cb7-4a8d-bebd-723b41dd5506 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:21.047044 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 218/424] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:21.050036 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515bc15a-3cb7-4a8d-bebd-723b41dd5506 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:21.058448 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb76726d-ccf2-4048-a7f1-71f153275011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:21.059111 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb76726d-ccf2-4048-a7f1-71f153275011 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:21.059807 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb76726d-ccf2-4048-a7f1-71f153275011 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:21.060116 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb76726d-ccf2-4048-a7f1-71f153275011 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:21.161272 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb76726d-ccf2-4048-a7f1-71f153275011 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:21.162930 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb76726d-ccf2-4048-a7f1-71f153275011 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:21.164558 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515bc15a-3cb7-4a8d-bebd-723b41dd5506 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:21.168630 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:21.215794 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66016521-d5be-4365-b29b-064b6bf974e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:21.216463 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66016521-d5be-4365-b29b-064b6bf974e2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:21.216748 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66016521-d5be-4365-b29b-064b6bf974e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:21.217029 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66016521-d5be-4365-b29b-064b6bf974e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:21.242418 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:21.301302 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66016521-d5be-4365-b29b-064b6bf974e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:21.302833 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66016521-d5be-4365-b29b-064b6bf974e2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:21.319123 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 207/425] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:06:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:06:21.321097 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd430c4-7d1d-46f9-873e-a57dd04a6e06 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:21.321846 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd430c4-7d1d-46f9-873e-a57dd04a6e06 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:21.322721 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd430c4-7d1d-46f9-873e-a57dd04a6e06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:21.342706 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-ddd430c4-7d1d-46f9-873e-a57dd04a6e06 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:21.343645 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ddd430c4-7d1d-46f9-873e-a57dd04a6e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:21.425530 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 208/426] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:06:21.426405 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd430c4-7d1d-46f9-873e-a57dd04a6e06 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:21.440811 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 219/427] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:06:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:06:21.447443 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66016521-d5be-4365-b29b-064b6bf974e2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:21.451059 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:21.538079 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:21.559608 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd430c4-7d1d-46f9-873e-a57dd04a6e06 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:21.584586 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66016521-d5be-4365-b29b-064b6bf974e2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:21.596564 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2e96d7-97bb-4e76-9498-891b6c095ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:21.597464 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e96d7-97bb-4e76-9498-891b6c095ff3 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:21.597776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e96d7-97bb-4e76-9498-891b6c095ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:21.598283 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e96d7-97bb-4e76-9498-891b6c095ff3 None admin] PATH_INFO: `/v3/projects/e192ac889bad428985088eea5ce750ab` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:21.599247 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2e96d7-97bb-4e76-9498-891b6c095ff3 None admin] RBAC: Authorizing `identity:get_project(project_id=e192ac889bad428985088eea5ce750ab)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:21.600909 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2e96d7-97bb-4e76-9498-891b6c095ff3 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:21.603879 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 209/428] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 4 09:06:21 2022] GET /identity/v3/projects/e192ac889bad428985088eea5ce750ab => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:21.604956 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2e96d7-97bb-4e76-9498-891b6c095ff3 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:21.626273 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae32c39-96b2-4e8e-a540-d95b6a95662a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:21.627361 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae32c39-96b2-4e8e-a540-d95b6a95662a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:21.627858 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae32c39-96b2-4e8e-a540-d95b6a95662a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:21.628355 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae32c39-96b2-4e8e-a540-d95b6a95662a None admin] PATH_INFO: `/v3/projects/e192ac889bad428985088eea5ce750ab` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:21.629790 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae32c39-96b2-4e8e-a540-d95b6a95662a None admin] RBAC: Authorizing `identity:get_project(project_id=e192ac889bad428985088eea5ce750ab)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:21.631500 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae32c39-96b2-4e8e-a540-d95b6a95662a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:21.637147 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 220/429] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 4 09:06:21 2022] GET /identity/v3/projects/e192ac889bad428985088eea5ce750ab => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:21.638244 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae32c39-96b2-4e8e-a540-d95b6a95662a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:21.715067 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2e96d7-97bb-4e76-9498-891b6c095ff3 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:21.716554 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:21.751013 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae32c39-96b2-4e8e-a540-d95b6a95662a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:21.754895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ee641d-1eaf-4c39-bad0-e7c81896352c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:21.755386 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee641d-1eaf-4c39-bad0-e7c81896352c None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:21.755776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee641d-1eaf-4c39-bad0-e7c81896352c None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:21.756178 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee641d-1eaf-4c39-bad0-e7c81896352c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:21.769236 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ee641d-1eaf-4c39-bad0-e7c81896352c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:06:21.777906 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-b6ee641d-1eaf-4c39-bad0-e7c81896352c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:06:21.777906 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:21.777906 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:21.777906 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:21.777906 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:21.777906 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:21.777906 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:21.777906 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:21.777906 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:21.777906 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:21.777906 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:21.777906 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:21.777906 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:06:21.777906 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:06:21.777906 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:06:21.777906 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:06:21.779030 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:06:21.779030 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:06:21.779030 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:06:21.779030 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:06:21.779030 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:21.779030 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:21.779030 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:21.779030 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:21.779030 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:21.779030 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:21.779030 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:21.779030 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:21.779030 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:21.779030 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:21.779030 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:21.779030 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:21.780073 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:21.780073 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:21.780073 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:21.780073 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:21.780073 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:06:21.780073 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:06:21.780073 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:06:21.780073 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:06:21.780073 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:06:21.780073 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:06:21.780073 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:06:21.780073 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:21.781708 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 210/430] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:06:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:21.787404 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ee641d-1eaf-4c39-bad0-e7c81896352c None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:21.795961 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760bc318-2c09-40cc-806b-562a31611cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:21.797150 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760bc318-2c09-40cc-806b-562a31611cf9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:21.797612 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760bc318-2c09-40cc-806b-562a31611cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:21.798071 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760bc318-2c09-40cc-806b-562a31611cf9 None admin] PATH_INFO: `/v3/groups/6e50e4b8168340a8bc57713559ff23ae` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:21.799455 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760bc318-2c09-40cc-806b-562a31611cf9 None admin] RBAC: Authorizing `identity:get_group(group_id=6e50e4b8168340a8bc57713559ff23ae)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:21.799696 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:21.801256 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760bc318-2c09-40cc-806b-562a31611cf9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:21.803225 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 221/431] 10.222.0.71 () {56 vars in 1180 bytes} [Mon Jul 4 09:06:21 2022] GET /identity/v3/groups/6e50e4b8168340a8bc57713559ff23ae => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:21.804301 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760bc318-2c09-40cc-806b-562a31611cf9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:21.900090 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ee641d-1eaf-4c39-bad0-e7c81896352c None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:21.909691 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760bc318-2c09-40cc-806b-562a31611cf9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:21.937437 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69facbf2-9688-4d3e-b18b-cd5a02b9ce9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:21.938552 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69facbf2-9688-4d3e-b18b-cd5a02b9ce9f None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:21.938988 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69facbf2-9688-4d3e-b18b-cd5a02b9ce9f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:21.939412 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69facbf2-9688-4d3e-b18b-cd5a02b9ce9f None admin] PATH_INFO: `/v3/groups/6e50e4b8168340a8bc57713559ff23ae` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:21.940799 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69facbf2-9688-4d3e-b18b-cd5a02b9ce9f None admin] RBAC: Authorizing `identity:get_group(group_id=6e50e4b8168340a8bc57713559ff23ae)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:21.942488 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69facbf2-9688-4d3e-b18b-cd5a02b9ce9f None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:21.944340 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 211/432] 10.222.0.71 () {56 vars in 1180 bytes} [Mon Jul 4 09:06:21 2022] GET /identity/v3/groups/6e50e4b8168340a8bc57713559ff23ae => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:21.945321 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69facbf2-9688-4d3e-b18b-cd5a02b9ce9f None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:21.954225 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141b5bbd-9b9e-4118-ac44-166d4e675411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:21.955196 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b5bbd-9b9e-4118-ac44-166d4e675411 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:21.955686 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b5bbd-9b9e-4118-ac44-166d4e675411 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:21.956185 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b5bbd-9b9e-4118-ac44-166d4e675411 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:21.957179 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141b5bbd-9b9e-4118-ac44-166d4e675411 None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:21.957691 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141b5bbd-9b9e-4118-ac44-166d4e675411 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:21.959889 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141b5bbd-9b9e-4118-ac44-166d4e675411 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:21.976155 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 222/433] 10.222.0.71 () {56 vars in 1101 bytes} [Mon Jul 4 09:06:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:21.977131 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141b5bbd-9b9e-4118-ac44-166d4e675411 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:21.980186 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:22.042014 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:22.069768 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69facbf2-9688-4d3e-b18b-cd5a02b9ce9f None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:22.084479 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141b5bbd-9b9e-4118-ac44-166d4e675411 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:22.111515 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eae8d0-f3fa-49f6-8681-385bf1a61a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:22.112666 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eae8d0-f3fa-49f6-8681-385bf1a61a16 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:22.113275 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eae8d0-f3fa-49f6-8681-385bf1a61a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:22.113777 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eae8d0-f3fa-49f6-8681-385bf1a61a16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:22.114828 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eae8d0-f3fa-49f6-8681-385bf1a61a16 None admin] RBAC: Adding query filter params (group.id=6e50e4b8168340a8bc57713559ff23ae, role.id=8692e99e9463459f9b7caa96b3051b93, scope.project.id=e192ac889bad428985088eea5ce750ab) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:22.115444 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eae8d0-f3fa-49f6-8681-385bf1a61a16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:22.117221 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eae8d0-f3fa-49f6-8681-385bf1a61a16 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:22.120850 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c49cbc-d766-4427-8980-2023c088e24c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:22.121521 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c49cbc-d766-4427-8980-2023c088e24c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:22.122070 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c49cbc-d766-4427-8980-2023c088e24c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:22.122363 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c49cbc-d766-4427-8980-2023c088e24c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:22.123199 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c49cbc-d766-4427-8980-2023c088e24c None admin] RBAC: Adding query filter params (group.id=6e50e4b8168340a8bc57713559ff23ae, role.id=5fb58444d5604bf689014e27dccacb45, scope.project.id=e192ac889bad428985088eea5ce750ab) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:22.123514 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c49cbc-d766-4427-8980-2023c088e24c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:22.124951 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c49cbc-d766-4427-8980-2023c088e24c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:22.125850 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 212/434] 10.222.0.71 () {56 vars in 1376 bytes} [Mon Jul 4 09:06:21 2022] GET /identity/v3/role_assignments?group.id=6e50e4b8168340a8bc57713559ff23ae&scope.project.id=e192ac889bad428985088eea5ce750ab&role.id=8692e99e9463459f9b7caa96b3051b93 => generated 611 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:22.127058 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eae8d0-f3fa-49f6-8681-385bf1a61a16 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:22.137193 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 223/435] 10.222.0.71 () {56 vars in 1376 bytes} [Mon Jul 4 09:06:21 2022] GET /identity/v3/role_assignments?group.id=6e50e4b8168340a8bc57713559ff23ae&scope.project.id=e192ac889bad428985088eea5ce750ab&role.id=5fb58444d5604bf689014e27dccacb45 => generated 611 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:22.256587 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eae8d0-f3fa-49f6-8681-385bf1a61a16 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:22.299994 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1506141-c722-43ea-a8ff-a80c97da658e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:22.300674 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1506141-c722-43ea-a8ff-a80c97da658e None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:22.300963 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1506141-c722-43ea-a8ff-a80c97da658e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:22.301281 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1506141-c722-43ea-a8ff-a80c97da658e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:22.313620 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:22.344072 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea48afef-e9bb-4055-9e6f-ea8384387764 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:22.344642 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea48afef-e9bb-4055-9e6f-ea8384387764 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:22.344642 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea48afef-e9bb-4055-9e6f-ea8384387764 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:22.346366 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 224/436] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:06:22.352966 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc03bfc-3fdb-4fa4-9d34-3fe9f8b2c8df None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:22.352966 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc03bfc-3fdb-4fa4-9d34-3fe9f8b2c8df None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:22.352966 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc03bfc-3fdb-4fa4-9d34-3fe9f8b2c8df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:22.372641 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2fc03bfc-3fdb-4fa4-9d34-3fe9f8b2c8df None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:22.373924 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2fc03bfc-3fdb-4fa4-9d34-3fe9f8b2c8df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:22.387306 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1506141-c722-43ea-a8ff-a80c97da658e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:22.389605 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1506141-c722-43ea-a8ff-a80c97da658e None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:22.452677 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:22.464740 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 225/437] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:06:22.495562 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc03bfc-3fdb-4fa4-9d34-3fe9f8b2c8df None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:22.513239 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 213/438] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:06:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:06:22.519160 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1506141-c722-43ea-a8ff-a80c97da658e None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:22.576763 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:22.619721 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc03bfc-3fdb-4fa4-9d34-3fe9f8b2c8df None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:22.655616 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1506141-c722-43ea-a8ff-a80c97da658e None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:22.671831 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d707be97-4455-43d2-9f10-2b0001810400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:22.673131 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d707be97-4455-43d2-9f10-2b0001810400 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:22.673131 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d707be97-4455-43d2-9f10-2b0001810400 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:22.673345 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d707be97-4455-43d2-9f10-2b0001810400 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:22.684004 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d707be97-4455-43d2-9f10-2b0001810400 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:06:22.691649 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-d707be97-4455-43d2-9f10-2b0001810400 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:06:22.691649 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:22.691649 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:22.691649 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:22.691649 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:22.691649 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:22.691649 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:22.691649 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:22.691649 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:22.691649 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:22.691649 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:22.691649 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:22.691649 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:06:22.691649 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:06:22.691649 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:06:22.691649 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:06:22.692769 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:06:22.692769 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:06:22.692769 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:06:22.692769 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:06:22.692769 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:22.692769 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:22.692769 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:22.692769 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:22.692769 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:22.692769 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:22.692769 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:22.692769 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:22.692769 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:22.692769 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:22.692769 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:22.692769 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:22.693690 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:22.693690 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:22.693690 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:22.693690 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:22.693690 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:06:22.693690 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:06:22.693690 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:06:22.693690 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:06:22.693690 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:06:22.693690 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:06:22.693690 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:06:22.693690 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:22.693690 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 226/439] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 4 09:06:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:06:22.694561 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e7cd5d-ee9b-4141-9984-3012d5d61937 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:22.695660 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e7cd5d-ee9b-4141-9984-3012d5d61937 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:22.695660 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e7cd5d-ee9b-4141-9984-3012d5d61937 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:22.698644 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 227/440] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:22.699671 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e7cd5d-ee9b-4141-9984-3012d5d61937 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:22.703643 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b79472-1f8f-4afb-ad9c-51ab1873915a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:22.705485 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b79472-1f8f-4afb-ad9c-51ab1873915a None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:22.705816 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b79472-1f8f-4afb-ad9c-51ab1873915a None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:22.706035 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b79472-1f8f-4afb-ad9c-51ab1873915a None admin] PATH_INFO: `/v3/projects/f8a33963b3504bf196b09f575de6e329` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:22.707283 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b79472-1f8f-4afb-ad9c-51ab1873915a None admin] RBAC: Authorizing `identity:get_project(project_id=f8a33963b3504bf196b09f575de6e329)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:22.708759 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b79472-1f8f-4afb-ad9c-51ab1873915a None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:22.712971 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 214/441] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 4 09:06:22 2022] GET /identity/v3/projects/f8a33963b3504bf196b09f575de6e329 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:06:22.714723 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a1bd45-35a9-457f-bf2c-6ebf6cc6567f None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:22.715031 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a1bd45-35a9-457f-bf2c-6ebf6cc6567f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:22.715483 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a1bd45-35a9-457f-bf2c-6ebf6cc6567f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:22.736831 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-52a1bd45-35a9-457f-bf2c-6ebf6cc6567f None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:22.738062 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-52a1bd45-35a9-457f-bf2c-6ebf6cc6567f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:22.767426 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:22.827151 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e7cd5d-ee9b-4141-9984-3012d5d61937 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:22.830801 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 215/442] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:06:22.831173 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a1bd45-35a9-457f-bf2c-6ebf6cc6567f None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:22.874012 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b852f08-3904-42b9-9237-211b8da362cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:22.874889 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b852f08-3904-42b9-9237-211b8da362cc None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:22.874999 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b852f08-3904-42b9-9237-211b8da362cc None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:22.875248 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b852f08-3904-42b9-9237-211b8da362cc None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:22.876721 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b852f08-3904-42b9-9237-211b8da362cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:22.876721 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b852f08-3904-42b9-9237-211b8da362cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:22.878537 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b852f08-3904-42b9-9237-211b8da362cc None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:22.882987 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:22.891992 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 228/443] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:06:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:06:22.893731 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b852f08-3904-42b9-9237-211b8da362cc None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:22.950121 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a1bd45-35a9-457f-bf2c-6ebf6cc6567f None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:22.983850 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce86274-04c9-40ed-ad14-e05642d7e7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:22.984708 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce86274-04c9-40ed-ad14-e05642d7e7d0 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:22.984991 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce86274-04c9-40ed-ad14-e05642d7e7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:22.985302 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce86274-04c9-40ed-ad14-e05642d7e7d0 None admin] PATH_INFO: `/v3/groups/6e50e4b8168340a8bc57713559ff23ae` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:22.986479 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce86274-04c9-40ed-ad14-e05642d7e7d0 None admin] RBAC: Authorizing `identity:get_group(group_id=6e50e4b8168340a8bc57713559ff23ae)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:22.987976 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce86274-04c9-40ed-ad14-e05642d7e7d0 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:22.989630 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 216/444] 10.222.0.71 () {56 vars in 1180 bytes} [Mon Jul 4 09:06:22 2022] GET /identity/v3/groups/6e50e4b8168340a8bc57713559ff23ae => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:22.990401 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce86274-04c9-40ed-ad14-e05642d7e7d0 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:22.998747 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b852f08-3904-42b9-9237-211b8da362cc None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:23.016387 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:23.035077 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65265397-ca50-4013-9e5f-32d130e920a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:23.035922 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65265397-ca50-4013-9e5f-32d130e920a2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:23.036276 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65265397-ca50-4013-9e5f-32d130e920a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:23.036626 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65265397-ca50-4013-9e5f-32d130e920a2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:23.046725 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65265397-ca50-4013-9e5f-32d130e920a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:06:23.054813 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-65265397-ca50-4013-9e5f-32d130e920a2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:06:23.054813 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:23.054813 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:23.054813 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:23.054813 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:23.054813 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:23.054813 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:23.054813 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:23.054813 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:23.054813 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:23.054813 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:23.054813 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:23.054813 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:06:23.054813 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:06:23.054813 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:06:23.054813 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:06:23.056745 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:06:23.056745 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:06:23.056745 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:06:23.056745 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:06:23.056745 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:23.056745 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:23.056745 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:23.056745 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:23.056745 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:23.056745 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:23.056745 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:23.056745 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:23.056745 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:23.056745 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:23.056745 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:23.056745 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:23.057590 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:23.057590 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:23.057590 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:23.057590 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:23.057590 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:06:23.057590 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:06:23.057590 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:06:23.057590 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:06:23.057590 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:06:23.057590 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:06:23.057590 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:06:23.057590 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:23.057590 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 229/445] 10.222.0.71 () {56 vars in 1114 bytes} [Mon Jul 4 09:06:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:23.057590 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65265397-ca50-4013-9e5f-32d130e920a2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:23.095288 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce86274-04c9-40ed-ad14-e05642d7e7d0 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:23.103065 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:23.130168 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fad728-73bb-444b-835d-361f233ef56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:23.130817 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fad728-73bb-444b-835d-361f233ef56d None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:23.131103 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fad728-73bb-444b-835d-361f233ef56d None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:23.131383 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fad728-73bb-444b-835d-361f233ef56d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:23.169883 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65265397-ca50-4013-9e5f-32d130e920a2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:23.203347 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16299dfc-44df-49ac-9606-1b50b8237dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:23.203996 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16299dfc-44df-49ac-9606-1b50b8237dfe None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:23.204279 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16299dfc-44df-49ac-9606-1b50b8237dfe None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:23.204553 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16299dfc-44df-49ac-9606-1b50b8237dfe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:23.205411 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16299dfc-44df-49ac-9606-1b50b8237dfe None admin] RBAC: Adding query filter params (group.id=6e50e4b8168340a8bc57713559ff23ae, role.id=8692e99e9463459f9b7caa96b3051b93, scope.project.id=f8a33963b3504bf196b09f575de6e329) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:23.205721 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16299dfc-44df-49ac-9606-1b50b8237dfe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:23.207138 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16299dfc-44df-49ac-9606-1b50b8237dfe None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:23.209763 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fad728-73bb-444b-835d-361f233ef56d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:23.211209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fad728-73bb-444b-835d-361f233ef56d None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:23.214235 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 230/446] 10.222.0.71 () {56 vars in 1376 bytes} [Mon Jul 4 09:06:23 2022] GET /identity/v3/role_assignments?group.id=6e50e4b8168340a8bc57713559ff23ae&scope.project.id=f8a33963b3504bf196b09f575de6e329&role.id=8692e99e9463459f9b7caa96b3051b93 => generated 611 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:23.214843 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16299dfc-44df-49ac-9606-1b50b8237dfe None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:23.276916 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:23.329791 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 217/447] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:06:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:06:23.335523 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fad728-73bb-444b-835d-361f233ef56d None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:23.344932 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:23.348491 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16299dfc-44df-49ac-9606-1b50b8237dfe None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:23.397489 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b7b31-7533-4273-80f0-8eede582afbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:23.397906 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b7b31-7533-4273-80f0-8eede582afbc None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:23.398355 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b7b31-7533-4273-80f0-8eede582afbc None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:23.398642 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b7b31-7533-4273-80f0-8eede582afbc None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:23.400239 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32b7b31-7533-4273-80f0-8eede582afbc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:23.400648 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32b7b31-7533-4273-80f0-8eede582afbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:23.402079 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32b7b31-7533-4273-80f0-8eede582afbc None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:23.413079 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 231/448] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 4 09:06:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:23.419126 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b7b31-7533-4273-80f0-8eede582afbc None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:23.460907 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fad728-73bb-444b-835d-361f233ef56d None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:23.503536 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4869b9a8-20c6-4129-a1a6-d0490a65b6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:23.503811 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4869b9a8-20c6-4129-a1a6-d0490a65b6ad None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:23.504098 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4869b9a8-20c6-4129-a1a6-d0490a65b6ad None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:23.504378 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4869b9a8-20c6-4129-a1a6-d0490a65b6ad None admin] PATH_INFO: `/v3/projects/a34f9b2c1cbf4bceb4067648553a5416` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:23.507450 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4869b9a8-20c6-4129-a1a6-d0490a65b6ad None admin] RBAC: Authorizing `identity:get_project(project_id=a34f9b2c1cbf4bceb4067648553a5416)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:23.507450 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4869b9a8-20c6-4129-a1a6-d0490a65b6ad None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:23.508820 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 218/449] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 4 09:06:23 2022] GET /identity/v3/projects/a34f9b2c1cbf4bceb4067648553a5416 => generated 344 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:06:23.514373 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4869b9a8-20c6-4129-a1a6-d0490a65b6ad None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:23.550020 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b7b31-7533-4273-80f0-8eede582afbc None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:23.553642 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:23.595036 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c221df-5466-4716-9e1f-f35e8c31d943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:23.595617 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c221df-5466-4716-9e1f-f35e8c31d943 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:23.595907 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c221df-5466-4716-9e1f-f35e8c31d943 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:23.596189 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c221df-5466-4716-9e1f-f35e8c31d943 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:23.607514 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:23.634188 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4869b9a8-20c6-4129-a1a6-d0490a65b6ad None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:23.663658 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c221df-5466-4716-9e1f-f35e8c31d943 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:23.665837 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c221df-5466-4716-9e1f-f35e8c31d943 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:23.669281 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee79ef9-76d8-446c-a736-b15a351f783d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:23.670135 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee79ef9-76d8-446c-a736-b15a351f783d None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:23.670463 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee79ef9-76d8-446c-a736-b15a351f783d None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:23.670754 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee79ef9-76d8-446c-a736-b15a351f783d None admin] PATH_INFO: `/v3/groups/7c24c12b114748848903409974efea20` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:23.671943 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee79ef9-76d8-446c-a736-b15a351f783d None admin] RBAC: Authorizing `identity:get_group(group_id=7c24c12b114748848903409974efea20)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:23.673417 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee79ef9-76d8-446c-a736-b15a351f783d None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:23.675398 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 219/450] 10.222.0.71 () {56 vars in 1180 bytes} [Mon Jul 4 09:06:23 2022] GET /identity/v3/groups/7c24c12b114748848903409974efea20 => generated 229 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:06:23.681287 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee79ef9-76d8-446c-a736-b15a351f783d None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:23.763979 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 232/451] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:06:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 04 09:06:23.769749 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c221df-5466-4716-9e1f-f35e8c31d943 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:23.772506 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:23.779218 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee79ef9-76d8-446c-a736-b15a351f783d None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:23.814110 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e88335-3dd8-4751-930e-dfa499f4c7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:23.814768 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e88335-3dd8-4751-930e-dfa499f4c7ee None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:23.815052 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e88335-3dd8-4751-930e-dfa499f4c7ee None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:23.815333 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e88335-3dd8-4751-930e-dfa499f4c7ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:23.816162 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e88335-3dd8-4751-930e-dfa499f4c7ee None admin] RBAC: Adding query filter params (group.id=7c24c12b114748848903409974efea20, role.id=f36e053b442540f08041ac7878760f65, scope.project.id=a34f9b2c1cbf4bceb4067648553a5416) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:23.816476 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e88335-3dd8-4751-930e-dfa499f4c7ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:23.817947 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e88335-3dd8-4751-930e-dfa499f4c7ee None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:23.823832 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 220/452] 10.222.0.71 () {56 vars in 1376 bytes} [Mon Jul 4 09:06:23 2022] GET /identity/v3/role_assignments?group.id=7c24c12b114748848903409974efea20&scope.project.id=a34f9b2c1cbf4bceb4067648553a5416&role.id=f36e053b442540f08041ac7878760f65 => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:06:23.872071 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c221df-5466-4716-9e1f-f35e8c31d943 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:23.907925 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6902c51-b486-476c-9d13-deb36e00df83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:23.908797 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6902c51-b486-476c-9d13-deb36e00df83 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:23.909110 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6902c51-b486-476c-9d13-deb36e00df83 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:23.909390 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6902c51-b486-476c-9d13-deb36e00df83 None admin] PATH_INFO: `/v3/projects/f8a33963b3504bf196b09f575de6e329` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:23.910611 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6902c51-b486-476c-9d13-deb36e00df83 None admin] RBAC: Authorizing `identity:get_project(project_id=f8a33963b3504bf196b09f575de6e329)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:23.912076 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6902c51-b486-476c-9d13-deb36e00df83 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:23.913850 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 233/453] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 4 09:06:23 2022] GET /identity/v3/projects/f8a33963b3504bf196b09f575de6e329 => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:23.919603 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e88335-3dd8-4751-930e-dfa499f4c7ee None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:23.928929 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:24.023588 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e88335-3dd8-4751-930e-dfa499f4c7ee None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:24.056884 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3a2a31-4ce1-409b-965d-636a851dcb66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:24.057763 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a2a31-4ce1-409b-965d-636a851dcb66 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:24.058055 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a2a31-4ce1-409b-965d-636a851dcb66 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:24.058331 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a2a31-4ce1-409b-965d-636a851dcb66 None admin] PATH_INFO: `/v3/groups/6e50e4b8168340a8bc57713559ff23ae` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:24.059485 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3a2a31-4ce1-409b-965d-636a851dcb66 None admin] RBAC: Authorizing `identity:get_group(group_id=6e50e4b8168340a8bc57713559ff23ae)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:24.060924 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3a2a31-4ce1-409b-965d-636a851dcb66 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:24.062572 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 221/454] 10.222.0.71 () {56 vars in 1180 bytes} [Mon Jul 4 09:06:23 2022] GET /identity/v3/groups/6e50e4b8168340a8bc57713559ff23ae => generated 226 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:24.068714 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6902c51-b486-476c-9d13-deb36e00df83 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:24.138095 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:24.166826 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6902c51-b486-476c-9d13-deb36e00df83 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:24.199721 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aec2ac-5120-4c7b-9e43-2cdcd7d489e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:24.200367 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aec2ac-5120-4c7b-9e43-2cdcd7d489e8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:24.200655 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aec2ac-5120-4c7b-9e43-2cdcd7d489e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:24.200938 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aec2ac-5120-4c7b-9e43-2cdcd7d489e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:24.201798 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aec2ac-5120-4c7b-9e43-2cdcd7d489e8 None admin] RBAC: Adding query filter params (group.id=6e50e4b8168340a8bc57713559ff23ae, role.id=5fb58444d5604bf689014e27dccacb45, scope.project.id=f8a33963b3504bf196b09f575de6e329) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:24.202114 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aec2ac-5120-4c7b-9e43-2cdcd7d489e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:24.203537 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aec2ac-5120-4c7b-9e43-2cdcd7d489e8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:24.209266 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 234/455] 10.222.0.71 () {56 vars in 1376 bytes} [Mon Jul 4 09:06:24 2022] GET /identity/v3/role_assignments?group.id=6e50e4b8168340a8bc57713559ff23ae&scope.project.id=f8a33963b3504bf196b09f575de6e329&role.id=5fb58444d5604bf689014e27dccacb45 => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:26.832214 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625784b1-4802-48c1-8333-b2833ad5abe2 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:26.833284 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625784b1-4802-48c1-8333-b2833ad5abe2 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:26.833784 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625784b1-4802-48c1-8333-b2833ad5abe2 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:26.835368 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 222/456] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:26.840863 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be0964-2a4f-48e7-a806-ade97ff6af20 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:26.841775 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be0964-2a4f-48e7-a806-ade97ff6af20 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:26.842275 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be0964-2a4f-48e7-a806-ade97ff6af20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:26.865112 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f8be0964-2a4f-48e7-a806-ade97ff6af20 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:26.866665 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f8be0964-2a4f-48e7-a806-ade97ff6af20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:26.959247 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 235/457] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:06:26.996902 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625784b1-4802-48c1-8333-b2833ad5abe2 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:27.102431 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:27.142673 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625784b1-4802-48c1-8333-b2833ad5abe2 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:27.192068 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf60076c-9912-4977-aad7-f277c2f5010e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:27.192869 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf60076c-9912-4977-aad7-f277c2f5010e None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:27.193184 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf60076c-9912-4977-aad7-f277c2f5010e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:27.193464 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf60076c-9912-4977-aad7-f277c2f5010e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:27.210382 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf60076c-9912-4977-aad7-f277c2f5010e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:27.212666 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf60076c-9912-4977-aad7-f277c2f5010e None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:27.234513 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-cf60076c-9912-4977-aad7-f277c2f5010e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:27.234513 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:27.234513 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:27.234513 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:27.234513 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:27.234513 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:27.234513 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:27.234513 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:27.234513 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:27.234513 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:27.234513 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:27.234513 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:27.234513 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:06:27.234513 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:06:27.234513 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:06:27.235434 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:06:27.235434 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:27.235434 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:27.235434 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:27.235434 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:27.235434 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:27.235434 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:27.235434 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:27.235434 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:27.235434 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:27.235434 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:27.235434 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:27.235434 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:27.235434 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:27.235434 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:27.235434 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:27.235434 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:27.236317 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:06:27.236317 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:06:27.236317 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:27.236317 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:27.238577 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 223/458] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:06:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:27.244557 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8be0964-2a4f-48e7-a806-ade97ff6af20 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:27.295613 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49219a7d-774d-4fc3-94f8-e5f1f8b7ddf3 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:27.296238 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49219a7d-774d-4fc3-94f8-e5f1f8b7ddf3 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:27.296715 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49219a7d-774d-4fc3-94f8-e5f1f8b7ddf3 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:27.298572 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 224/459] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:27.304069 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc56ab-a324-4624-a744-3267d5aedc7d None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:27.304588 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc56ab-a324-4624-a744-3267d5aedc7d None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:27.305097 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc56ab-a324-4624-a744-3267d5aedc7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:27.323009 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-edcc56ab-a324-4624-a744-3267d5aedc7d None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:27.324639 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-edcc56ab-a324-4624-a744-3267d5aedc7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:27.349884 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:27.388550 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8be0964-2a4f-48e7-a806-ade97ff6af20 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:27.411413 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 225/460] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:06:27.414948 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6057fd-bfe8-4e6c-b035-4e3fd92791d5 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:27.415870 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6057fd-bfe8-4e6c-b035-4e3fd92791d5 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:27.416599 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6057fd-bfe8-4e6c-b035-4e3fd92791d5 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:27.422456 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 226/461] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:27 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:27.422456 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30d95a-77df-46f2-b8d4-890d4a235aee None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:27.422656 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30d95a-77df-46f2-b8d4-890d4a235aee None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:27.422940 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30d95a-77df-46f2-b8d4-890d4a235aee None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:27.424229 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 227/462] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:27.427761 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b1a7df-7418-4f3c-aa3f-fe37b9f3835e None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:27.427761 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b1a7df-7418-4f3c-aa3f-fe37b9f3835e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:27.427761 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b1a7df-7418-4f3c-aa3f-fe37b9f3835e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:27.438958 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d909ba-6c23-49de-8364-8b2a65eeaeb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:27.439641 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d909ba-6c23-49de-8364-8b2a65eeaeb1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:27.439927 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d909ba-6c23-49de-8364-8b2a65eeaeb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:27.440206 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d909ba-6c23-49de-8364-8b2a65eeaeb1 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:27.440935 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d909ba-6c23-49de-8364-8b2a65eeaeb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:27.441271 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d909ba-6c23-49de-8364-8b2a65eeaeb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:27.443084 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d909ba-6c23-49de-8364-8b2a65eeaeb1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:27.448899 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-e3b1a7df-7418-4f3c-aa3f-fe37b9f3835e None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:27.450130 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e3b1a7df-7418-4f3c-aa3f-fe37b9f3835e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:27.466914 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 236/463] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:06:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:27.466914 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d54153d-61de-4141-a8d8-372f28ccdd0f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:27.466914 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d54153d-61de-4141-a8d8-372f28ccdd0f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:27.466914 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d54153d-61de-4141-a8d8-372f28ccdd0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:27.482219 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1d54153d-61de-4141-a8d8-372f28ccdd0f None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:27.483496 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1d54153d-61de-4141-a8d8-372f28ccdd0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:27.514342 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:27.535311 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 228/464] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:06:27.536469 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b1a7df-7418-4f3c-aa3f-fe37b9f3835e None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:27.563024 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 237/465] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:06:27.573352 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d54153d-61de-4141-a8d8-372f28ccdd0f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:27.630949 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:27.657862 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b1a7df-7418-4f3c-aa3f-fe37b9f3835e None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:27.682676 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d54153d-61de-4141-a8d8-372f28ccdd0f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:27.698358 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762faa18-6e36-4553-a730-386442788e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:27.699155 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762faa18-6e36-4553-a730-386442788e58 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:27.699452 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762faa18-6e36-4553-a730-386442788e58 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:27.699707 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762faa18-6e36-4553-a730-386442788e58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:27.710268 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762faa18-6e36-4553-a730-386442788e58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:27.712703 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762faa18-6e36-4553-a730-386442788e58 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:27.718410 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0624c59-e525-4086-963c-87197b566494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:27.719087 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0624c59-e525-4086-963c-87197b566494 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:27.719783 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0624c59-e525-4086-963c-87197b566494 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:27.720227 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0624c59-e525-4086-963c-87197b566494 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:27.721360 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0624c59-e525-4086-963c-87197b566494 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:27.722797 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0624c59-e525-4086-963c-87197b566494 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:27.731207 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-762faa18-6e36-4553-a730-386442788e58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:27.731207 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:27.731207 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:27.731207 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:27.731207 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:27.731207 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:27.731207 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:27.731207 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:27.731207 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:27.731207 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:27.731207 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:27.731207 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:27.731207 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:06:27.731207 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:06:27.731207 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:06:27.732243 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:06:27.732243 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:27.732243 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:27.732243 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:27.732243 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:27.732243 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:27.732243 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:27.732243 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:27.732243 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:27.732243 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:27.732243 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:27.732243 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:27.732243 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:27.732243 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:27.732243 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:27.732243 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:27.732243 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:27.733579 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:06:27.733579 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:06:27.733579 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:27.733579 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:27.733579 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 229/466] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:06:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:27.733579 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762faa18-6e36-4553-a730-386442788e58 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:27.750350 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 238/467] 10.222.0.71 () {60 vars in 1132 bytes} [Mon Jul 4 09:06:27 2022] POST /identity/v3/users => generated 246 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:06:27.761691 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0624c59-e525-4086-963c-87197b566494 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:27.782315 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:27.872443 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762faa18-6e36-4553-a730-386442788e58 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:27.918781 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0c395c-9879-4cff-a3c7-a76b24534898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:27.919520 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c395c-9879-4cff-a3c7-a76b24534898 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:27.919808 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c395c-9879-4cff-a3c7-a76b24534898 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:27.920090 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c395c-9879-4cff-a3c7-a76b24534898 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:27.937252 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0c395c-9879-4cff-a3c7-a76b24534898 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:27.938981 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0c395c-9879-4cff-a3c7-a76b24534898 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:27.939619 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0624c59-e525-4086-963c-87197b566494 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:27.956666 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:27.961078 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-fb0c395c-9879-4cff-a3c7-a76b24534898 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:27.961078 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:27.961078 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:27.961078 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:27.961078 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:27.961078 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:27.961078 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:27.961078 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:27.961078 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:27.961078 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:27.961078 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:27.961078 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:27.961078 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:06:27.961078 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:06:27.961078 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:06:27.962254 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:06:27.962254 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:27.962254 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:27.962254 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:27.962254 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:27.962254 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:27.962254 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:27.962254 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:27.962254 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:27.962254 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:27.962254 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:27.962254 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:27.962254 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:27.962254 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:27.962254 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:27.962254 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:27.962254 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:27.963153 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:06:27.963153 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:06:27.963153 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:27.963153 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:27.963363 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 230/468] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:06:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:27.969664 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0c395c-9879-4cff-a3c7-a76b24534898 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:27.985568 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b84288-5436-4d66-8ba4-cb04e8276ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:27.986386 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b84288-5436-4d66-8ba4-cb04e8276ad1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:27.986676 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b84288-5436-4d66-8ba4-cb04e8276ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:27.986963 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b84288-5436-4d66-8ba4-cb04e8276ad1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:27.999034 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b84288-5436-4d66-8ba4-cb04e8276ad1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:28.000499 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b84288-5436-4d66-8ba4-cb04e8276ad1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:28.019355 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-77b84288-5436-4d66-8ba4-cb04e8276ad1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:28.019355 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:28.019355 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:28.019355 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:28.019355 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:28.019355 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:28.019355 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:28.019355 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:28.019355 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:28.019355 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:28.019355 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:28.019355 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:28.019355 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:06:28.019355 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:06:28.019355 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:06:28.020460 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:06:28.020460 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:28.020460 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:28.020460 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:28.020460 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:28.020460 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:28.020460 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:28.020460 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:28.020460 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:28.020460 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:28.020460 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:28.020460 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:28.020460 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:28.020460 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:28.020460 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:28.020460 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:28.020460 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:28.021401 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:06:28.021401 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:06:28.021401 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:28.021401 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:28.021401 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 239/469] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:06:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:28.021845 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b84288-5436-4d66-8ba4-cb04e8276ad1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:28.089786 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:28.101866 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0c395c-9879-4cff-a3c7-a76b24534898 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:28.140732 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab5f6ad-24f3-45be-8647-035c78825536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:28.141778 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5f6ad-24f3-45be-8647-035c78825536 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:28.142295 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5f6ad-24f3-45be-8647-035c78825536 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:28.142774 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5f6ad-24f3-45be-8647-035c78825536 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:28.143831 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab5f6ad-24f3-45be-8647-035c78825536 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:28.144343 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab5f6ad-24f3-45be-8647-035c78825536 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:28.145963 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab5f6ad-24f3-45be-8647-035c78825536 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:28.148169 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b84288-5436-4d66-8ba4-cb04e8276ad1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:28.159858 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 231/470] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:06:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:28.161115 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab5f6ad-24f3-45be-8647-035c78825536 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:28.191123 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d52b83-83e9-4175-81a0-b41823eecd4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:28.191717 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d52b83-83e9-4175-81a0-b41823eecd4c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:28.192004 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d52b83-83e9-4175-81a0-b41823eecd4c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:28.192286 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d52b83-83e9-4175-81a0-b41823eecd4c None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:28.193021 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d52b83-83e9-4175-81a0-b41823eecd4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:28.193371 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d52b83-83e9-4175-81a0-b41823eecd4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:28.194966 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d52b83-83e9-4175-81a0-b41823eecd4c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:28.213612 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 240/471] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:06:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:28.213612 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d52b83-83e9-4175-81a0-b41823eecd4c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:28.242485 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:28.278932 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab5f6ad-24f3-45be-8647-035c78825536 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:28.314934 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf6ee8b-0876-433b-92d3-5b2f8580f5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:28.315606 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6ee8b-0876-433b-92d3-5b2f8580f5c3 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:28.315894 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6ee8b-0876-433b-92d3-5b2f8580f5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:28.316178 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6ee8b-0876-433b-92d3-5b2f8580f5c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:28.316904 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf6ee8b-0876-433b-92d3-5b2f8580f5c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:28.317244 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf6ee8b-0876-433b-92d3-5b2f8580f5c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:28.318636 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf6ee8b-0876-433b-92d3-5b2f8580f5c3 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:28.326597 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d52b83-83e9-4175-81a0-b41823eecd4c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:28.336340 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 232/472] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:06:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:28.336340 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf6ee8b-0876-433b-92d3-5b2f8580f5c3 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:28.348458 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:28.365377 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40798aa-7b0f-4650-861b-b1c992d4212e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:28.366061 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40798aa-7b0f-4650-861b-b1c992d4212e None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:28.366348 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40798aa-7b0f-4650-861b-b1c992d4212e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:28.366784 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40798aa-7b0f-4650-861b-b1c992d4212e None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:28.367333 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40798aa-7b0f-4650-861b-b1c992d4212e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:28.369345 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40798aa-7b0f-4650-861b-b1c992d4212e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:28.392073 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 241/473] 10.222.0.71 () {60 vars in 1132 bytes} [Mon Jul 4 09:06:28 2022] POST /identity/v3/users => generated 248 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:06:28.392968 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90069fa-86c8-492d-bb42-2d4de1ba0b1c None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:28.393290 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90069fa-86c8-492d-bb42-2d4de1ba0b1c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:28.393579 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90069fa-86c8-492d-bb42-2d4de1ba0b1c None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:28.394939 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 242/474] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:28.400395 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90069fa-86c8-492d-bb42-2d4de1ba0b1c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:28.483537 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf6ee8b-0876-433b-92d3-5b2f8580f5c3 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:28.500986 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:28.536686 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90069fa-86c8-492d-bb42-2d4de1ba0b1c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:28.542274 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8079cf-2bc0-4961-a63a-986f651669a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:28.543016 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8079cf-2bc0-4961-a63a-986f651669a1 None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:28.543307 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8079cf-2bc0-4961-a63a-986f651669a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:28.543587 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8079cf-2bc0-4961-a63a-986f651669a1 None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:28.544954 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8079cf-2bc0-4961-a63a-986f651669a1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:28.546562 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8079cf-2bc0-4961-a63a-986f651669a1 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:28.573081 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 233/475] 10.222.0.71 () {60 vars in 1132 bytes} [Mon Jul 4 09:06:28 2022] POST /identity/v3/users => generated 249 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:06:28.573657 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389534f3-7bee-4f54-8760-03bdeccc4c1e None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:28.573951 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389534f3-7bee-4f54-8760-03bdeccc4c1e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:28.574233 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389534f3-7bee-4f54-8760-03bdeccc4c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:28.589230 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a74a2a-5f94-4d48-98a9-2f27ab6bbc16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:28.591038 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a74a2a-5f94-4d48-98a9-2f27ab6bbc16 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:28.591499 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a74a2a-5f94-4d48-98a9-2f27ab6bbc16 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:28.592091 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a74a2a-5f94-4d48-98a9-2f27ab6bbc16 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:28.593184 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a74a2a-5f94-4d48-98a9-2f27ab6bbc16 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:28.594886 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a74a2a-5f94-4d48-98a9-2f27ab6bbc16 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:28.596250 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-389534f3-7bee-4f54-8760-03bdeccc4c1e None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:28.597601 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-389534f3-7bee-4f54-8760-03bdeccc4c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:28.624209 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 243/476] 10.222.0.71 () {60 vars in 1132 bytes} [Mon Jul 4 09:06:28 2022] POST /identity/v3/users => generated 248 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:06:28.685960 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:28.693205 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 234/477] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:06:28.715793 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a74a2a-5f94-4d48-98a9-2f27ab6bbc16 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:28.849612 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a74a2a-5f94-4d48-98a9-2f27ab6bbc16 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:28.885560 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd44c78e-1d41-4628-8ddd-a88c6051fc37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:28.886241 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd44c78e-1d41-4628-8ddd-a88c6051fc37 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:28.886527 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd44c78e-1d41-4628-8ddd-a88c6051fc37 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:28.886806 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd44c78e-1d41-4628-8ddd-a88c6051fc37 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:28.887565 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd44c78e-1d41-4628-8ddd-a88c6051fc37 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:28.888930 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd44c78e-1d41-4628-8ddd-a88c6051fc37 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:28.892518 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:28.897198 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.sql.core [None req-bd44c78e-1d41-4628-8ddd-a88c6051fc37 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 04 09:06:28.897198 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 04 09:06:28.897198 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [parameters: {'id': '3045d15a9bf64673a0efdb4acf26eee5', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 04 09:06:28.897198 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80170) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 04 09:06:28.917583 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-bd44c78e-1d41-4628-8ddd-a88c6051fc37 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 04 09:06:28.917583 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:28.917583 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 04 09:06:28.917583 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 04 09:06:28.917583 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 04 09:06:28.917583 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 04 09:06:28.917583 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 04 09:06:28.917583 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result = self._query(query) Jul 04 09:06:28.917583 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 04 09:06:28.917583 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application conn.query(q) Jul 04 09:06:28.917583 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 04 09:06:28.917583 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 04 09:06:28.917583 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 04 09:06:28.917583 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result.read() Jul 04 09:06:28.917583 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 04 09:06:28.918575 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 04 09:06:28.918575 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 04 09:06:28.918575 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 04 09:06:28.918575 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 04 09:06:28.918575 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 04 09:06:28.918575 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 04 09:06:28.918575 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 04 09:06:28.918575 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 04 09:06:28.918575 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:28.918575 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 04 09:06:28.918575 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:28.918575 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:28.918575 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 04 09:06:28.918575 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 04 09:06:28.918575 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 04 09:06:28.918575 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return ref.to_dict() Jul 04 09:06:28.918575 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 04 09:06:28.918575 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application next(self.gen) Jul 04 09:06:28.919791 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 04 09:06:28.919791 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application yield resource Jul 04 09:06:28.919791 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 04 09:06:28.919791 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application next(self.gen) Jul 04 09:06:28.919791 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 04 09:06:28.919791 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.session.rollback() Jul 04 09:06:28.919791 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 04 09:06:28.919791 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.force_reraise() Jul 04 09:06:28.919791 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 04 09:06:28.919791 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise self.value Jul 04 09:06:28.919791 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 04 09:06:28.919791 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 04 09:06:28.919791 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 04 09:06:28.919791 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application session.commit() Jul 04 09:06:28.919791 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 04 09:06:28.919791 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 04 09:06:28.919791 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 04 09:06:28.920656 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._prepare_impl() Jul 04 09:06:28.920656 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 04 09:06:28.920656 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.session.flush() Jul 04 09:06:28.920656 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 04 09:06:28.920656 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._flush(objects) Jul 04 09:06:28.920656 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 04 09:06:28.920656 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 04 09:06:28.920656 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 04 09:06:28.920656 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application compat.raise_( Jul 04 09:06:28.920656 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 04 09:06:28.920656 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception Jul 04 09:06:28.920656 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 04 09:06:28.920656 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application flush_context.execute() Jul 04 09:06:28.920656 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 04 09:06:28.920656 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rec.execute(self) Jul 04 09:06:28.920656 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 04 09:06:28.920656 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 04 09:06:28.921585 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 04 09:06:28.921585 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 04 09:06:28.921585 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 04 09:06:28.921585 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 04 09:06:28.921585 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 04 09:06:28.921585 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 04 09:06:28.921585 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 04 09:06:28.921585 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 04 09:06:28.921585 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 04 09:06:28.921585 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 04 09:06:28.921585 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 04 09:06:28.921585 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 04 09:06:28.921585 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 04 09:06:28.921585 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 04 09:06:28.921585 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 04 09:06:28.921585 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception Jul 04 09:06:28.922579 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 04 09:06:28.922579 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 04 09:06:28.922579 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 04 09:06:28.922579 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 04 09:06:28.922579 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 04 09:06:28.922579 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result = self._query(query) Jul 04 09:06:28.922579 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 04 09:06:28.922579 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application conn.query(q) Jul 04 09:06:28.922579 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 04 09:06:28.922579 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 04 09:06:28.922579 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 04 09:06:28.922579 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result.read() Jul 04 09:06:28.922579 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 04 09:06:28.922579 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 04 09:06:28.922579 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 04 09:06:28.922579 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 04 09:06:28.923433 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 04 09:06:28.923433 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 04 09:06:28.923433 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 04 09:06:28.923433 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 04 09:06:28.923433 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 04 09:06:28.923433 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 04 09:06:28.923433 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [parameters: {'id': '3045d15a9bf64673a0efdb4acf26eee5', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 04 09:06:28.923433 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 04 09:06:28.923433 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:28.923433 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:06:28.923433 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:28.923433 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:28.923433 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:28.923433 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:28.923433 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:28.923433 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:28.924230 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:28.924230 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:28.924230 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:28.924230 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:28.924230 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:28.924230 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:28.924230 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 04 09:06:28.924230 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 04 09:06:28.924230 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:28.924230 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:28.924230 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 04 09:06:28.924230 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 04 09:06:28.924230 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 04 09:06:28.924230 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 04 09:06:28.924230 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 04 09:06:28.924230 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:28.925069 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 244/478] 10.222.0.71 () {60 vars in 1131 bytes} [Mon Jul 4 09:06:28 2022] POST /identity/v3/roles => generated 144 bytes in 204 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:28.928054 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389534f3-7bee-4f54-8760-03bdeccc4c1e None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:29.047459 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389534f3-7bee-4f54-8760-03bdeccc4c1e None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:29.092236 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c0e330-c5db-4b17-8420-ffed3cdfc1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:29.093376 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c0e330-c5db-4b17-8420-ffed3cdfc1c3 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:29.093812 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c0e330-c5db-4b17-8420-ffed3cdfc1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:29.094263 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c0e330-c5db-4b17-8420-ffed3cdfc1c3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:29.113236 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c0e330-c5db-4b17-8420-ffed3cdfc1c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:06:29.122476 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-49c0e330-c5db-4b17-8420-ffed3cdfc1c3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:06:29.122476 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:29.122476 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:29.122476 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:29.122476 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:29.122476 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:29.122476 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:29.122476 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:29.122476 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:29.122476 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:29.122476 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:29.122476 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:29.122476 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:06:29.122476 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:06:29.122476 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:06:29.122476 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:06:29.123626 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:06:29.123626 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:06:29.123626 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:06:29.123626 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:06:29.123626 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:29.123626 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:29.123626 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:29.123626 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:29.123626 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:29.123626 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:29.123626 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:29.123626 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:29.123626 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:29.123626 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:29.123626 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:29.123626 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:29.124658 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:29.124658 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:29.124658 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:29.124658 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:29.124658 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:06:29.124658 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:06:29.124658 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:06:29.124658 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:06:29.124658 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:06:29.124658 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:06:29.124658 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:06:29.124658 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:29.127284 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 235/479] 10.222.0.71 () {56 vars in 1114 bytes} [Mon Jul 4 09:06:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:29.132578 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd44c78e-1d41-4628-8ddd-a88c6051fc37 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:29.278344 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd44c78e-1d41-4628-8ddd-a88c6051fc37 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:29.325615 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2abc479-b322-4a5c-8a99-2665b3cc82ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:29.326302 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2abc479-b322-4a5c-8a99-2665b3cc82ed None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:29.326590 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2abc479-b322-4a5c-8a99-2665b3cc82ed None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:29.326871 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2abc479-b322-4a5c-8a99-2665b3cc82ed None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:29.327632 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2abc479-b322-4a5c-8a99-2665b3cc82ed None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:29.327948 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2abc479-b322-4a5c-8a99-2665b3cc82ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:29.329339 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2abc479-b322-4a5c-8a99-2665b3cc82ed None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:29.339303 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 245/480] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 4 09:06:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:30.158699 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55382b98-f341-4562-807a-7cf3df4d25d6 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:30.159576 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55382b98-f341-4562-807a-7cf3df4d25d6 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:30.160038 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55382b98-f341-4562-807a-7cf3df4d25d6 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:30.161882 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 236/481] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:30.167497 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46a002-1658-484b-8e33-9623f3725c7e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:30.168283 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46a002-1658-484b-8e33-9623f3725c7e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:30.168712 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46a002-1658-484b-8e33-9623f3725c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:30.190089 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4b46a002-1658-484b-8e33-9623f3725c7e None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:30.191486 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4b46a002-1658-484b-8e33-9623f3725c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:30.264593 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:30.277840 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 246/482] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:06:30.338754 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55382b98-f341-4562-807a-7cf3df4d25d6 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:30.417254 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:30.470690 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55382b98-f341-4562-807a-7cf3df4d25d6 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:30.517778 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cde3f95-65be-4657-8208-3e64e7d476c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:30.518586 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cde3f95-65be-4657-8208-3e64e7d476c9 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:30.518874 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cde3f95-65be-4657-8208-3e64e7d476c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:30.519157 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cde3f95-65be-4657-8208-3e64e7d476c9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:30.536079 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cde3f95-65be-4657-8208-3e64e7d476c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:06:30.544674 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-2cde3f95-65be-4657-8208-3e64e7d476c9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:06:30.544674 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:30.544674 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:30.544674 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:30.544674 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:30.544674 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:30.544674 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:30.544674 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:30.544674 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:30.544674 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:30.544674 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:30.544674 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:30.544674 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:06:30.544674 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:06:30.544674 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:06:30.544674 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:06:30.545700 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:06:30.545700 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:06:30.545700 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:06:30.545700 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:06:30.545700 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:30.545700 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:30.545700 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:30.545700 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:30.545700 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:30.545700 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:30.545700 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:30.545700 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:30.545700 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:30.545700 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:30.545700 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:30.545700 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:30.546523 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:30.546523 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:30.546523 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:30.546523 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:30.546523 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:06:30.546523 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:06:30.546523 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:06:30.546523 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:06:30.546523 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:06:30.546523 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:06:30.546523 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:06:30.546523 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:30.552521 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 237/483] 10.222.0.71 () {56 vars in 1102 bytes} [Mon Jul 4 09:06:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:30.553568 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b46a002-1658-484b-8e33-9623f3725c7e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:30.680003 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b46a002-1658-484b-8e33-9623f3725c7e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:30.728777 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83ab55c-e56f-4fcb-be38-c289306bb1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:30.729484 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ab55c-e56f-4fcb-be38-c289306bb1f6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:30.729772 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ab55c-e56f-4fcb-be38-c289306bb1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:30.730050 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ab55c-e56f-4fcb-be38-c289306bb1f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:30.730806 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83ab55c-e56f-4fcb-be38-c289306bb1f6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:30.731118 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83ab55c-e56f-4fcb-be38-c289306bb1f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:30.732492 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83ab55c-e56f-4fcb-be38-c289306bb1f6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:30.741175 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 247/484] 10.222.0.71 () {56 vars in 1103 bytes} [Mon Jul 4 09:06:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:30.750128 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cde3f95-65be-4657-8208-3e64e7d476c9 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:30.887743 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cde3f95-65be-4657-8208-3e64e7d476c9 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:30.904690 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:30.912333 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d095597-c971-4280-96e6-4eecc16d555a None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:30.912629 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d095597-c971-4280-96e6-4eecc16d555a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:30.912919 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d095597-c971-4280-96e6-4eecc16d555a None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:30.915260 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 248/485] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:30.925612 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c424fa-a73b-4086-80d4-1289dfbf055a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:30.925907 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c424fa-a73b-4086-80d4-1289dfbf055a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:30.926222 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c424fa-a73b-4086-80d4-1289dfbf055a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:30.929269 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e10e4-5f3a-4c19-8364-a15e0d4aa75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:30.930881 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e10e4-5f3a-4c19-8364-a15e0d4aa75d None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:30.931504 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e10e4-5f3a-4c19-8364-a15e0d4aa75d None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:30.931928 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e10e4-5f3a-4c19-8364-a15e0d4aa75d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:30.943787 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1e10e4-5f3a-4c19-8364-a15e0d4aa75d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:30.945480 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1e10e4-5f3a-4c19-8364-a15e0d4aa75d None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:30.962628 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-32c424fa-a73b-4086-80d4-1289dfbf055a None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:30.963855 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-32c424fa-a73b-4086-80d4-1289dfbf055a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:30.964378 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-1c1e10e4-5f3a-4c19-8364-a15e0d4aa75d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:30.964378 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:30.964378 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:30.964378 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:30.964378 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:30.964378 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:30.964378 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:30.964378 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:30.964378 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:30.964378 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:30.964378 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:30.964378 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:30.964378 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:06:30.964378 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:06:30.964378 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:06:30.965400 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:06:30.965400 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:30.965400 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:30.965400 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:30.965400 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:30.965400 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:30.965400 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:30.965400 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:30.965400 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:30.965400 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:30.965400 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:30.965400 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:30.965400 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:30.965400 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:30.965400 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:30.965400 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:30.965400 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:30.966462 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:06:30.966462 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:06:30.966462 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:30.966462 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:30.968358 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 238/486] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:06:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:30.978748 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e10e4-5f3a-4c19-8364-a15e0d4aa75d None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:31.036408 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:31.067652 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 249/487] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:06:31.099187 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c424fa-a73b-4086-80d4-1289dfbf055a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:31.146650 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e10e4-5f3a-4c19-8364-a15e0d4aa75d None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:31.204564 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c424fa-a73b-4086-80d4-1289dfbf055a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:31.214788 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c417e-b31e-43b0-8e84-caa3341f05e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:31.216055 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c417e-b31e-43b0-8e84-caa3341f05e4 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:31.216481 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c417e-b31e-43b0-8e84-caa3341f05e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:31.216899 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c417e-b31e-43b0-8e84-caa3341f05e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:31.217787 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47c417e-b31e-43b0-8e84-caa3341f05e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:31.218336 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47c417e-b31e-43b0-8e84-caa3341f05e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:31.219949 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47c417e-b31e-43b0-8e84-caa3341f05e4 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:31.243210 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 239/488] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:06:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:31.244943 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013fba08-9642-4866-bcfb-3c7f844eb997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:31.245782 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013fba08-9642-4866-bcfb-3c7f844eb997 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:31.246069 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013fba08-9642-4866-bcfb-3c7f844eb997 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:31.246350 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013fba08-9642-4866-bcfb-3c7f844eb997 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:31.249202 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c417e-b31e-43b0-8e84-caa3341f05e4 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:31.267470 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013fba08-9642-4866-bcfb-3c7f844eb997 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:06:31.277392 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-013fba08-9642-4866-bcfb-3c7f844eb997 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:06:31.277392 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:31.277392 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:31.277392 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:31.277392 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:31.277392 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:31.277392 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:31.277392 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:31.277392 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:31.277392 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:31.277392 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:31.277392 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:31.277392 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:06:31.277392 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:06:31.277392 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:06:31.277392 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:06:31.278334 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:06:31.278334 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:06:31.278334 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:06:31.278334 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:06:31.278334 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:31.278334 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:31.278334 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:31.278334 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:31.278334 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:31.278334 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:31.278334 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:31.278334 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:31.278334 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:31.278334 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:31.278334 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:31.278334 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:31.279155 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:31.279155 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:31.279155 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:31.279155 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:31.279155 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:06:31.279155 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:06:31.279155 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:06:31.279155 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:06:31.279155 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:06:31.279155 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:06:31.279155 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:06:31.279155 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:31.279155 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 250/489] 10.222.0.71 () {56 vars in 1102 bytes} [Mon Jul 4 09:06:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:06:31.292550 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013fba08-9642-4866-bcfb-3c7f844eb997 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:31.303352 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:31.353658 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:31.405168 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013fba08-9642-4866-bcfb-3c7f844eb997 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:31.436240 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e206cf8-ae25-4ba7-9864-0a38d7bddd3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:31.436905 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e206cf8-ae25-4ba7-9864-0a38d7bddd3c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:31.437224 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e206cf8-ae25-4ba7-9864-0a38d7bddd3c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:31.437508 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e206cf8-ae25-4ba7-9864-0a38d7bddd3c None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:31.438262 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e206cf8-ae25-4ba7-9864-0a38d7bddd3c None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:31.438573 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e206cf8-ae25-4ba7-9864-0a38d7bddd3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:31.439994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e206cf8-ae25-4ba7-9864-0a38d7bddd3c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:31.449329 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 251/490] 10.222.0.71 () {56 vars in 1103 bytes} [Mon Jul 4 09:06:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:06:31.454876 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e206cf8-ae25-4ba7-9864-0a38d7bddd3c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:31.490662 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c417e-b31e-43b0-8e84-caa3341f05e4 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:31.533279 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9cb751-d735-42d6-a1fb-4f619e07a185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:31.534823 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9cb751-d735-42d6-a1fb-4f619e07a185 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:31.535298 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9cb751-d735-42d6-a1fb-4f619e07a185 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:31.536167 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9cb751-d735-42d6-a1fb-4f619e07a185 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:31.546256 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9cb751-d735-42d6-a1fb-4f619e07a185 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:31.548505 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9cb751-d735-42d6-a1fb-4f619e07a185 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:31.558322 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e206cf8-ae25-4ba7-9864-0a38d7bddd3c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:31.565808 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:31.566135 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-8e9cb751-d735-42d6-a1fb-4f619e07a185 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 04 09:06:31.566135 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:31.566135 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:31.566135 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:31.566135 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:31.566135 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:31.566135 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:31.566135 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:31.566135 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:31.566135 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:31.566135 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:31.566135 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:31.566135 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:06:31.566135 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:06:31.566135 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:06:31.566135 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:06:31.567252 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:31.567252 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:31.567252 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:06:31.567252 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:06:31.567252 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:06:31.567252 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:06:31.567252 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:31.567252 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:31.567252 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:31.567252 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:31.567252 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:31.567252 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:31.567252 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:31.567252 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:31.567252 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:31.567252 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:31.567252 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:31.567252 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:31.568195 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:31.568195 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:31.568195 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:06:31.568195 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:06:31.568195 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:06:31.568195 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:06:31.568195 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:06:31.568195 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:06:31.568195 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 04 09:06:31.568195 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:31.568195 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 240/491] 10.222.0.71 () {56 vars in 1093 bytes} [Mon Jul 4 09:06:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:31.568774 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8e785-dec8-4b9a-aed4-a479c14ab0c8 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:31.568840 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8e785-dec8-4b9a-aed4-a479c14ab0c8 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:31.569469 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8e785-dec8-4b9a-aed4-a479c14ab0c8 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:31.571412 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 241/492] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:31.573078 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e42f9d-5c21-4adf-b44f-467a9e61f801 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:31.573078 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e42f9d-5c21-4adf-b44f-467a9e61f801 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:31.573271 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e42f9d-5c21-4adf-b44f-467a9e61f801 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:31.578185 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 242/493] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:31.579361 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab2d36c-cb1a-4932-8a8f-e2029b3a3870 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:31.579647 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab2d36c-cb1a-4932-8a8f-e2029b3a3870 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:31.579945 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab2d36c-cb1a-4932-8a8f-e2029b3a3870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:31.597072 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-0ab2d36c-cb1a-4932-8a8f-e2029b3a3870 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:31.598191 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0ab2d36c-cb1a-4932-8a8f-e2029b3a3870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:31.601746 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b38c50a-927c-4fe8-aef4-8b851070fad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:31.602601 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b38c50a-927c-4fe8-aef4-8b851070fad9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:31.602885 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b38c50a-927c-4fe8-aef4-8b851070fad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:31.603163 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b38c50a-927c-4fe8-aef4-8b851070fad9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:31.615490 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b38c50a-927c-4fe8-aef4-8b851070fad9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:31.616955 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b38c50a-927c-4fe8-aef4-8b851070fad9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:31.636483 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-5b38c50a-927c-4fe8-aef4-8b851070fad9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:31.636483 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:31.636483 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:31.636483 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:31.636483 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:31.636483 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:31.636483 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:31.636483 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:31.636483 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:31.636483 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:31.636483 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:31.636483 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:31.636483 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:06:31.636483 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:06:31.636483 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:06:31.637618 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:06:31.637618 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:31.637618 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:31.637618 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:31.637618 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:31.637618 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:31.637618 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:31.637618 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:31.637618 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:31.637618 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:31.637618 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:31.637618 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:31.637618 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:31.637618 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:31.637618 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:31.637618 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:31.637618 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:31.643249 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:06:31.643249 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:06:31.643249 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:31.643249 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:31.643249 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 252/494] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:06:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:31.643249 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b38c50a-927c-4fe8-aef4-8b851070fad9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:31.672959 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 243/495] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:06:31.684875 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a5117e-1792-4ea7-ba57-6b229eb6bf8b None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:31.684875 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a5117e-1792-4ea7-ba57-6b229eb6bf8b None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:31.684875 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a5117e-1792-4ea7-ba57-6b229eb6bf8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:31.699624 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-99a5117e-1792-4ea7-ba57-6b229eb6bf8b None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:31.700921 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-99a5117e-1792-4ea7-ba57-6b229eb6bf8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:31.715946 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:31.753032 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b38c50a-927c-4fe8-aef4-8b851070fad9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:31.785064 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 244/496] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:06:31.785064 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a5117e-1792-4ea7-ba57-6b229eb6bf8b None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:31.806563 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e186cf67-7084-4ee8-8e4f-559eedba724a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:31.807273 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e186cf67-7084-4ee8-8e4f-559eedba724a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:31.807565 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e186cf67-7084-4ee8-8e4f-559eedba724a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:31.807845 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e186cf67-7084-4ee8-8e4f-559eedba724a None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:31.808687 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e186cf67-7084-4ee8-8e4f-559eedba724a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:31.808997 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e186cf67-7084-4ee8-8e4f-559eedba724a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:31.810475 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e186cf67-7084-4ee8-8e4f-559eedba724a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:31.853405 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 253/497] 10.222.0.71 () {56 vars in 1133 bytes} [Mon Jul 4 09:06:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:31.854169 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e186cf67-7084-4ee8-8e4f-559eedba724a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:31.873058 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:31.886716 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a5117e-1792-4ea7-ba57-6b229eb6bf8b None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:31.922218 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8959a94a-2da1-4a7b-ae3c-37ef0709cef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:31.922883 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959a94a-2da1-4a7b-ae3c-37ef0709cef5 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:31.923169 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959a94a-2da1-4a7b-ae3c-37ef0709cef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:31.923449 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959a94a-2da1-4a7b-ae3c-37ef0709cef5 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:31.924169 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8959a94a-2da1-4a7b-ae3c-37ef0709cef5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:31.924478 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8959a94a-2da1-4a7b-ae3c-37ef0709cef5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:31.925907 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8959a94a-2da1-4a7b-ae3c-37ef0709cef5 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:31.938113 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 245/498] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:06:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:31.938113 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8959a94a-2da1-4a7b-ae3c-37ef0709cef5 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:31.951502 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:31.972660 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e186cf67-7084-4ee8-8e4f-559eedba724a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:32.007670 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85a5a4d-dfac-4885-917e-b3e9841660fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:32.009177 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a5a4d-dfac-4885-917e-b3e9841660fb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:32.009469 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a5a4d-dfac-4885-917e-b3e9841660fb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:32.009753 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a5a4d-dfac-4885-917e-b3e9841660fb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:32.020380 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85a5a4d-dfac-4885-917e-b3e9841660fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:06:32.029238 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-a85a5a4d-dfac-4885-917e-b3e9841660fb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:06:32.029238 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:32.029238 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:32.029238 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:32.029238 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:32.029238 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:32.029238 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:32.029238 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:32.029238 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:32.029238 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:32.029238 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:32.029238 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:32.029238 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:06:32.029238 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:06:32.029238 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:06:32.029238 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:06:32.030181 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:06:32.030181 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:06:32.030181 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:06:32.030181 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:06:32.030181 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:32.030181 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:32.030181 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:32.030181 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:32.030181 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:32.030181 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:32.030181 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:32.030181 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:32.030181 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:32.030181 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:32.030181 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:32.030181 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:32.030986 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:32.030986 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:32.030986 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:32.030986 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:32.030986 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:06:32.030986 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:06:32.030986 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:06:32.030986 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:06:32.030986 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:06:32.030986 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:06:32.030986 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:06:32.030986 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:32.030986 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 254/499] 10.222.0.71 () {56 vars in 1102 bytes} [Mon Jul 4 09:06:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:32.031664 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85a5a4d-dfac-4885-917e-b3e9841660fb None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:32.055928 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8959a94a-2da1-4a7b-ae3c-37ef0709cef5 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:32.094691 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95da92d1-8d01-445f-87d8-fea610976f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:32.095575 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95da92d1-8d01-445f-87d8-fea610976f06 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:32.095934 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95da92d1-8d01-445f-87d8-fea610976f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:32.096287 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95da92d1-8d01-445f-87d8-fea610976f06 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:32.107445 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95da92d1-8d01-445f-87d8-fea610976f06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:06:32.116334 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-95da92d1-8d01-445f-87d8-fea610976f06 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:06:32.116334 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:32.116334 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:32.116334 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:32.116334 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:32.116334 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:32.116334 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:32.116334 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:32.116334 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:32.116334 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:32.116334 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:32.116334 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:32.116334 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:06:32.116334 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:06:32.116334 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:06:32.116334 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:06:32.117522 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:06:32.117522 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:06:32.117522 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:06:32.117522 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:06:32.117522 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:32.117522 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:32.117522 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:32.117522 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:32.117522 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:32.117522 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:32.117522 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:32.117522 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:32.117522 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:32.117522 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:32.117522 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:32.117522 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:32.118395 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:32.118395 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:32.118395 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:32.118395 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:32.118395 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:06:32.118395 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:06:32.118395 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:06:32.118395 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:06:32.118395 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:06:32.118395 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:06:32.118395 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:06:32.118395 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:32.118395 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 246/500] 10.222.0.71 () {56 vars in 1102 bytes} [Mon Jul 4 09:06:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:32.119077 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95da92d1-8d01-445f-87d8-fea610976f06 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:32.119912 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:32.145845 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85a5a4d-dfac-4885-917e-b3e9841660fb None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:32.179912 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32488cf0-29a9-43c4-ac45-17f5ab5e6229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:32.180605 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32488cf0-29a9-43c4-ac45-17f5ab5e6229 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:32.180890 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32488cf0-29a9-43c4-ac45-17f5ab5e6229 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:32.181215 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32488cf0-29a9-43c4-ac45-17f5ab5e6229 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:32.188965 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:32.223274 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95da92d1-8d01-445f-87d8-fea610976f06 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:32.248770 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32488cf0-29a9-43c4-ac45-17f5ab5e6229 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:32.250270 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32488cf0-29a9-43c4-ac45-17f5ab5e6229 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:32.256928 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4feeb4-24d0-45ae-bcc1-5d9a95495322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:32.257751 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4feeb4-24d0-45ae-bcc1-5d9a95495322 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:32.258039 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4feeb4-24d0-45ae-bcc1-5d9a95495322 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:32.258344 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4feeb4-24d0-45ae-bcc1-5d9a95495322 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:32.265788 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4feeb4-24d0-45ae-bcc1-5d9a95495322 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:32.267229 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4feeb4-24d0-45ae-bcc1-5d9a95495322 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:32.281798 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-eb4feeb4-24d0-45ae-bcc1-5d9a95495322 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 04 09:06:32.281798 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:32.281798 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:32.281798 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:32.281798 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:32.281798 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:32.281798 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:32.281798 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:32.281798 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:32.281798 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:32.281798 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:32.281798 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:32.281798 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:06:32.281798 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:06:32.281798 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:06:32.281798 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:06:32.282731 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:32.282731 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:32.282731 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:06:32.282731 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:06:32.282731 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:06:32.282731 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:06:32.282731 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:32.282731 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:32.282731 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:32.282731 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:32.282731 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:32.282731 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:32.282731 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:32.282731 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:32.282731 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:32.282731 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:32.282731 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:32.282731 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:32.283642 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:32.283642 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:32.283642 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:06:32.283642 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:06:32.283642 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:06:32.283642 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:06:32.283642 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:06:32.283642 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:06:32.283642 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 04 09:06:32.283642 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:32.283642 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 247/501] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:06:32 2022] GET /identity/v3/users/glance => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:32.284440 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4feeb4-24d0-45ae-bcc1-5d9a95495322 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:32.327700 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:32.353453 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 255/502] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:06:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:06:32.354289 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32488cf0-29a9-43c4-ac45-17f5ab5e6229 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:32.382454 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4feeb4-24d0-45ae-bcc1-5d9a95495322 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:32.409964 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:32.419281 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f301994-a2ec-417a-9e22-9533513ba204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:32.419951 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f301994-a2ec-417a-9e22-9533513ba204 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:32.420240 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f301994-a2ec-417a-9e22-9533513ba204 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:32.420509 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f301994-a2ec-417a-9e22-9533513ba204 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:32.421281 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f301994-a2ec-417a-9e22-9533513ba204 None admin] RBAC: Adding query filter params (name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:32.421589 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f301994-a2ec-417a-9e22-9533513ba204 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:32.422954 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f301994-a2ec-417a-9e22-9533513ba204 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:32.431516 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 248/503] 10.222.0.71 () {56 vars in 1103 bytes} [Mon Jul 4 09:06:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:32.432808 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ad740-892a-4df1-ad9e-450e1679bfc2 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:32.433139 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ad740-892a-4df1-ad9e-450e1679bfc2 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:32.433420 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ad740-892a-4df1-ad9e-450e1679bfc2 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:32.434632 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 249/504] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:32.435312 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ad740-892a-4df1-ad9e-450e1679bfc2 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:32.461230 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32488cf0-29a9-43c4-ac45-17f5ab5e6229 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:32.498141 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804d5380-426c-45ef-96ee-102c8765ad47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:32.498737 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d5380-426c-45ef-96ee-102c8765ad47 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:32.499025 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d5380-426c-45ef-96ee-102c8765ad47 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:32.499307 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d5380-426c-45ef-96ee-102c8765ad47 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:32.500053 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804d5380-426c-45ef-96ee-102c8765ad47 None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:32.500365 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804d5380-426c-45ef-96ee-102c8765ad47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:32.501758 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804d5380-426c-45ef-96ee-102c8765ad47 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:32.509858 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 256/505] 10.222.0.71 () {56 vars in 1103 bytes} [Mon Jul 4 09:06:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:32.510793 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804d5380-426c-45ef-96ee-102c8765ad47 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:32.541410 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ad740-892a-4df1-ad9e-450e1679bfc2 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:32.550218 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:32.575740 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f840c7-3fa6-4ccc-ae36-2111ac1f633c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:32.576413 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f840c7-3fa6-4ccc-ae36-2111ac1f633c None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:32.576704 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f840c7-3fa6-4ccc-ae36-2111ac1f633c None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:32.576983 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f840c7-3fa6-4ccc-ae36-2111ac1f633c None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:32.577858 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f840c7-3fa6-4ccc-ae36-2111ac1f633c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:32.578173 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f840c7-3fa6-4ccc-ae36-2111ac1f633c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:32.579668 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f840c7-3fa6-4ccc-ae36-2111ac1f633c None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:32.610429 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804d5380-426c-45ef-96ee-102c8765ad47 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:32.616046 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 250/506] 10.222.0.71 () {56 vars in 1137 bytes} [Mon Jul 4 09:06:32 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:32.616875 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f840c7-3fa6-4ccc-ae36-2111ac1f633c None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:32.644391 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93f722a-e469-4b15-9613-e87cbad7b0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:32.645206 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f722a-e469-4b15-9613-e87cbad7b0a9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:32.645492 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f722a-e469-4b15-9613-e87cbad7b0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:32.645766 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f722a-e469-4b15-9613-e87cbad7b0a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:32.655815 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93f722a-e469-4b15-9613-e87cbad7b0a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:32.657326 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93f722a-e469-4b15-9613-e87cbad7b0a9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:32.666444 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:32.675803 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-d93f722a-e469-4b15-9613-e87cbad7b0a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:32.675803 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:32.675803 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:32.675803 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:32.675803 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:32.675803 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:32.675803 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:32.675803 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:32.675803 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:32.675803 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:32.675803 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:32.675803 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:32.675803 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:06:32.675803 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:06:32.675803 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:06:32.676799 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:06:32.676799 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:32.676799 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:32.676799 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:32.676799 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:32.676799 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:32.676799 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:32.676799 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:32.676799 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:32.676799 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:32.676799 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:32.676799 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:32.676799 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:32.676799 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:32.676799 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:32.676799 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:32.676799 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:32.677806 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:06:32.677806 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:06:32.677806 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:32.677806 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:32.677806 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 257/507] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:06:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:32.678521 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5039e2-b0b8-4f47-9e4b-5f9f538c1def None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:32.678810 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5039e2-b0b8-4f47-9e4b-5f9f538c1def None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:32.679092 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5039e2-b0b8-4f47-9e4b-5f9f538c1def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:32.697385 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-dc5039e2-b0b8-4f47-9e4b-5f9f538c1def None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:32.698715 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dc5039e2-b0b8-4f47-9e4b-5f9f538c1def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:32.726552 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f840c7-3fa6-4ccc-ae36-2111ac1f633c None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:32.763692 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786ba6a0-90a9-4098-9f93-a3fa86cdfecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:32.764454 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786ba6a0-90a9-4098-9f93-a3fa86cdfecf None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:32.764745 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786ba6a0-90a9-4098-9f93-a3fa86cdfecf None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:32.765062 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786ba6a0-90a9-4098-9f93-a3fa86cdfecf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:32.766472 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 258/508] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:06:32.767303 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5039e2-b0b8-4f47-9e4b-5f9f538c1def None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:32.775809 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786ba6a0-90a9-4098-9f93-a3fa86cdfecf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:32.777325 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786ba6a0-90a9-4098-9f93-a3fa86cdfecf None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:32.795901 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-786ba6a0-90a9-4098-9f93-a3fa86cdfecf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:32.795901 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:32.795901 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:32.795901 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:32.795901 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:32.795901 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:32.795901 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:32.795901 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:32.795901 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:32.795901 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:32.795901 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:32.795901 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:32.795901 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:06:32.795901 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:06:32.795901 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:06:32.796972 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:06:32.796972 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:32.796972 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:32.796972 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:32.796972 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:32.796972 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:32.796972 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:32.796972 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:32.796972 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:32.796972 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:32.796972 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:32.796972 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:32.796972 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:32.796972 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:32.796972 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:32.796972 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:32.796972 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:32.798837 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:06:32.798837 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:06:32.798837 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:32.798837 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:32.799111 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:32.801600 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 251/509] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:06:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:32.802441 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786ba6a0-90a9-4098-9f93-a3fa86cdfecf None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:32.876957 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5039e2-b0b8-4f47-9e4b-5f9f538c1def None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:32.899765 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:32.902808 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786ba6a0-90a9-4098-9f93-a3fa86cdfecf None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:32.912402 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8d05de-82e2-458e-b40d-b7fa9e9e4ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:32.913239 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d05de-82e2-458e-b40d-b7fa9e9e4ce0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:32.913525 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d05de-82e2-458e-b40d-b7fa9e9e4ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:32.913806 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d05de-82e2-458e-b40d-b7fa9e9e4ce0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:32.924591 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8d05de-82e2-458e-b40d-b7fa9e9e4ce0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:32.926139 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8d05de-82e2-458e-b40d-b7fa9e9e4ce0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:32.937138 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9556469-a24c-475d-91c6-28be533f330f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:32.937786 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9556469-a24c-475d-91c6-28be533f330f None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:32.938076 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9556469-a24c-475d-91c6-28be533f330f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:32.938381 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9556469-a24c-475d-91c6-28be533f330f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:32.943230 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-cd8d05de-82e2-458e-b40d-b7fa9e9e4ce0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:32.943230 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:32.943230 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:32.943230 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:32.943230 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:32.943230 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:32.943230 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:32.943230 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:32.943230 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:32.943230 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:32.943230 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:32.943230 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:32.943230 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:06:32.943230 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:06:32.943230 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:06:32.944226 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:06:32.944226 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:32.944226 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:32.944226 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:32.944226 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:32.944226 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:32.944226 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:32.944226 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:32.944226 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:32.944226 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:32.944226 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:32.944226 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:32.944226 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:32.944226 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:32.944226 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:32.944226 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:32.944226 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:32.945268 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:06:32.945268 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:06:32.945268 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:32.945268 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:32.945268 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 259/510] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:06:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:32.945268 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8d05de-82e2-458e-b40d-b7fa9e9e4ce0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:33.010323 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9556469-a24c-475d-91c6-28be533f330f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:33.011819 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9556469-a24c-475d-91c6-28be533f330f None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:33.021322 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:33.058298 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8d05de-82e2-458e-b40d-b7fa9e9e4ce0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:33.095567 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a90b74-2fbf-4c55-97b8-5015e96f357e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:33.096237 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a90b74-2fbf-4c55-97b8-5015e96f357e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:33.096521 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a90b74-2fbf-4c55-97b8-5015e96f357e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:33.096804 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a90b74-2fbf-4c55-97b8-5015e96f357e None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:33.097563 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a90b74-2fbf-4c55-97b8-5015e96f357e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:33.097873 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a90b74-2fbf-4c55-97b8-5015e96f357e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:33.099252 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a90b74-2fbf-4c55-97b8-5015e96f357e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:33.110932 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 260/511] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:06:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:33.111723 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a90b74-2fbf-4c55-97b8-5015e96f357e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:33.115714 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:33.122357 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 252/512] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:06:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:06:33.123380 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9556469-a24c-475d-91c6-28be533f330f None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:33.212421 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a90b74-2fbf-4c55-97b8-5015e96f357e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:33.221934 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9556469-a24c-475d-91c6-28be533f330f None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:33.240428 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:33.246001 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169e2077-9679-4209-a10f-210dd538f411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:33.246790 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e2077-9679-4209-a10f-210dd538f411 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:33.247074 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e2077-9679-4209-a10f-210dd538f411 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:33.247351 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e2077-9679-4209-a10f-210dd538f411 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:33.255676 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5b55dd-bb3a-4db4-830b-6db0316003a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:33.256421 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b55dd-bb3a-4db4-830b-6db0316003a8 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:33.256711 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b55dd-bb3a-4db4-830b-6db0316003a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:33.256991 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b55dd-bb3a-4db4-830b-6db0316003a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:33.257754 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5b55dd-bb3a-4db4-830b-6db0316003a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:33.257936 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169e2077-9679-4209-a10f-210dd538f411 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:33.258794 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5b55dd-bb3a-4db4-830b-6db0316003a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:33.259939 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5b55dd-bb3a-4db4-830b-6db0316003a8 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:33.260053 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169e2077-9679-4209-a10f-210dd538f411 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:33.270923 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 253/513] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:06:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:33.271755 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5b55dd-bb3a-4db4-830b-6db0316003a8 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:33.277796 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-169e2077-9679-4209-a10f-210dd538f411 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:33.277796 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:33.277796 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:33.277796 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:33.277796 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:33.277796 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:33.277796 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:33.277796 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:33.277796 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:33.277796 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:33.277796 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:33.277796 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:33.277796 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:06:33.277796 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:06:33.277796 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:06:33.278849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:06:33.278849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:33.278849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:33.278849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:33.278849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:33.278849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:33.278849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:33.278849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:33.278849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:33.278849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:33.278849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:33.278849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:33.278849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:33.278849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:33.278849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:33.278849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:33.278849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:33.280043 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:06:33.280043 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:06:33.280043 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:33.280043 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:33.280043 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 261/514] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:06:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:33.280043 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169e2077-9679-4209-a10f-210dd538f411 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:33.337361 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:33.379520 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5b55dd-bb3a-4db4-830b-6db0316003a8 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:33.397462 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169e2077-9679-4209-a10f-210dd538f411 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:33.418397 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4dad72-ecc0-40a7-ac72-34df04763f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:33.419066 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4dad72-ecc0-40a7-ac72-34df04763f0d None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:33.419348 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4dad72-ecc0-40a7-ac72-34df04763f0d None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:33.419627 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4dad72-ecc0-40a7-ac72-34df04763f0d None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:33.420347 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4dad72-ecc0-40a7-ac72-34df04763f0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:33.420658 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4dad72-ecc0-40a7-ac72-34df04763f0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:33.422091 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4dad72-ecc0-40a7-ac72-34df04763f0d None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:33.431079 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4516feda-17b5-4cff-893a-007582ce87cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:33.431868 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4516feda-17b5-4cff-893a-007582ce87cc None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:33.432158 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4516feda-17b5-4cff-893a-007582ce87cc None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:33.432444 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4516feda-17b5-4cff-893a-007582ce87cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:33.433721 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 254/515] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:06:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:33.434509 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4dad72-ecc0-40a7-ac72-34df04763f0d None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:33.439675 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4516feda-17b5-4cff-893a-007582ce87cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:33.441157 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4516feda-17b5-4cff-893a-007582ce87cc None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:33.456545 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-4516feda-17b5-4cff-893a-007582ce87cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:06:33.456545 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:33.456545 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:33.456545 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:33.456545 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:33.456545 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:33.456545 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:33.456545 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:33.456545 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:33.456545 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:33.456545 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:33.456545 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:33.456545 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:06:33.456545 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:06:33.456545 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:06:33.457560 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:06:33.457560 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:33.457560 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:33.457560 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:33.457560 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:33.457560 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:33.457560 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:33.457560 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:33.457560 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:33.457560 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:33.457560 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:33.457560 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:33.457560 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:33.457560 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:33.457560 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:33.457560 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:33.457560 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:33.458428 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:06:33.458428 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:06:33.458428 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:06:33.458428 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:06:33.458428 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:06:33.458428 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:06:33.458428 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:06:33.458428 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:33.458428 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 262/516] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 4 09:06:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:33.458924 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4516feda-17b5-4cff-893a-007582ce87cc None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:33.505948 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:33.539350 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4dad72-ecc0-40a7-ac72-34df04763f0d None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:33.559761 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4516feda-17b5-4cff-893a-007582ce87cc None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:33.560245 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:33.574410 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f267950f-5807-4bff-8330-4f3b8d4cff97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:33.575206 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f267950f-5807-4bff-8330-4f3b8d4cff97 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:33.575484 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f267950f-5807-4bff-8330-4f3b8d4cff97 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:33.575754 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f267950f-5807-4bff-8330-4f3b8d4cff97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:33.585693 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f267950f-5807-4bff-8330-4f3b8d4cff97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:33.588481 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f267950f-5807-4bff-8330-4f3b8d4cff97 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:33.595183 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcafbb81-616f-4d5a-b3f4-8cd619916bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:33.595966 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcafbb81-616f-4d5a-b3f4-8cd619916bdf None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:33.596249 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcafbb81-616f-4d5a-b3f4-8cd619916bdf None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:33.596525 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcafbb81-616f-4d5a-b3f4-8cd619916bdf None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:33.604619 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcafbb81-616f-4d5a-b3f4-8cd619916bdf None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:33.606076 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcafbb81-616f-4d5a-b3f4-8cd619916bdf None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:33.607139 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-f267950f-5807-4bff-8330-4f3b8d4cff97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:33.607139 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:33.607139 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:33.607139 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:33.607139 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:33.607139 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:33.607139 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:33.607139 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:33.607139 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:33.607139 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:33.607139 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:33.607139 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:33.607139 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:06:33.607139 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:06:33.607139 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:06:33.608062 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:06:33.608062 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:33.608062 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:33.608062 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:33.608062 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:33.608062 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:33.608062 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:33.608062 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:33.608062 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:33.608062 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:33.608062 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:33.608062 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:33.608062 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:33.608062 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:33.608062 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:33.608062 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:33.608062 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:33.608921 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:06:33.608921 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:06:33.608921 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:33.608921 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:33.608921 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 255/517] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:06:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:33.609228 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f267950f-5807-4bff-8330-4f3b8d4cff97 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:33.622421 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-fcafbb81-616f-4d5a-b3f4-8cd619916bdf None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 04 09:06:33.622421 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:33.622421 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:33.622421 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:33.622421 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:33.622421 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:33.622421 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:33.622421 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:33.622421 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:33.622421 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:33.622421 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:33.622421 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:33.622421 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:06:33.622421 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:06:33.622421 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:06:33.622421 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:06:33.623465 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:33.623465 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:33.623465 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:06:33.623465 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:06:33.623465 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:06:33.623465 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:06:33.623465 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:33.623465 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:33.623465 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:33.623465 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:33.623465 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:33.623465 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:33.623465 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:33.623465 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:33.623465 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:33.623465 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:33.623465 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:33.623465 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:33.624367 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:33.624367 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:33.624367 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:06:33.624367 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:06:33.624367 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:06:33.624367 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:06:33.624367 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:06:33.624367 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:06:33.624367 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 04 09:06:33.624367 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:33.624367 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 263/518] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:06:33 2022] GET /identity/v3/users/cinder => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:33.624367 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcafbb81-616f-4d5a-b3f4-8cd619916bdf None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:33.717469 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f267950f-5807-4bff-8330-4f3b8d4cff97 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:33.727599 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcafbb81-616f-4d5a-b3f4-8cd619916bdf None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:33.729911 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:33.755499 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f9bd84-dd10-432a-9057-eda9967cf68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:33.756169 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f9bd84-dd10-432a-9057-eda9967cf68b None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:33.756457 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f9bd84-dd10-432a-9057-eda9967cf68b None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:33.756746 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f9bd84-dd10-432a-9057-eda9967cf68b None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:33.757502 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f9bd84-dd10-432a-9057-eda9967cf68b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:33.757814 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f9bd84-dd10-432a-9057-eda9967cf68b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:33.759353 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f9bd84-dd10-432a-9057-eda9967cf68b None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:33.761526 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a691e6d-a51b-44b3-892c-a5684e4fe4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:33.762321 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a691e6d-a51b-44b3-892c-a5684e4fe4a0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:33.762609 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a691e6d-a51b-44b3-892c-a5684e4fe4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:33.762888 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a691e6d-a51b-44b3-892c-a5684e4fe4a0 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:33.770272 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a691e6d-a51b-44b3-892c-a5684e4fe4a0 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:33.770468 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 256/519] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:06:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:33.771205 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f9bd84-dd10-432a-9057-eda9967cf68b None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:33.771725 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a691e6d-a51b-44b3-892c-a5684e4fe4a0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:33.788333 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-4a691e6d-a51b-44b3-892c-a5684e4fe4a0 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 04 09:06:33.788333 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:33.788333 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:33.788333 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:33.788333 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:33.788333 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:33.788333 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:33.788333 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:33.788333 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:33.788333 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:33.788333 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:33.788333 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:33.788333 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:06:33.788333 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:06:33.788333 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:06:33.788333 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:06:33.790273 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:33.790273 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:33.790273 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:06:33.790273 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:06:33.790273 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:06:33.790273 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:06:33.790273 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:33.790273 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:33.790273 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:33.790273 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:33.790273 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:33.790273 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:33.790273 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:33.790273 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:33.790273 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:33.790273 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:33.790273 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:33.790273 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:33.791411 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:33.791411 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:33.791411 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:06:33.791411 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:06:33.791411 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:06:33.791411 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:06:33.791411 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:06:33.791411 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:06:33.791411 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 04 09:06:33.791411 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:33.791411 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 264/520] 10.222.0.71 () {56 vars in 1102 bytes} [Mon Jul 4 09:06:33 2022] GET /identity/v3/users/neutron => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:33.791411 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a691e6d-a51b-44b3-892c-a5684e4fe4a0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:33.808540 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:33.878175 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f9bd84-dd10-432a-9057-eda9967cf68b None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:33.890635 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a691e6d-a51b-44b3-892c-a5684e4fe4a0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:33.911932 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7accbee8-7e56-44c0-8e72-355553b615c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:33.912588 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7accbee8-7e56-44c0-8e72-355553b615c9 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:33.912870 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7accbee8-7e56-44c0-8e72-355553b615c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:33.913170 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7accbee8-7e56-44c0-8e72-355553b615c9 None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:33.913957 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7accbee8-7e56-44c0-8e72-355553b615c9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:33.914262 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7accbee8-7e56-44c0-8e72-355553b615c9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:33.916352 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7accbee8-7e56-44c0-8e72-355553b615c9 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:33.925425 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6814dea6-f615-43b8-8d52-564d225b9d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:33.926095 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6814dea6-f615-43b8-8d52-564d225b9d44 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:33.926384 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6814dea6-f615-43b8-8d52-564d225b9d44 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:33.926689 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6814dea6-f615-43b8-8d52-564d225b9d44 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:33.927390 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6814dea6-f615-43b8-8d52-564d225b9d44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:33.927700 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6814dea6-f615-43b8-8d52-564d225b9d44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:33.929074 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6814dea6-f615-43b8-8d52-564d225b9d44 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:33.933787 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 257/521] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 4 09:06:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:33.934584 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7accbee8-7e56-44c0-8e72-355553b615c9 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:33.940518 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 265/522] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:06:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:33.941940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6814dea6-f615-43b8-8d52-564d225b9d44 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:33.968833 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:34.028994 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:34.038599 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7accbee8-7e56-44c0-8e72-355553b615c9 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:34.062614 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6814dea6-f615-43b8-8d52-564d225b9d44 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:34.081295 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f767d5-a39f-49f6-87ce-f0a37076e1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:34.081966 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f767d5-a39f-49f6-87ce-f0a37076e1f7 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:34.082254 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f767d5-a39f-49f6-87ce-f0a37076e1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:34.082536 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f767d5-a39f-49f6-87ce-f0a37076e1f7 None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:34.083367 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f767d5-a39f-49f6-87ce-f0a37076e1f7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:34.083688 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f767d5-a39f-49f6-87ce-f0a37076e1f7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:34.085172 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f767d5-a39f-49f6-87ce-f0a37076e1f7 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:34.097973 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d2ea96-2e5c-4e69-8d8b-a8882a476b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:34.098644 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d2ea96-2e5c-4e69-8d8b-a8882a476b19 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:34.098932 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d2ea96-2e5c-4e69-8d8b-a8882a476b19 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:34.099213 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d2ea96-2e5c-4e69-8d8b-a8882a476b19 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:34.099925 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d2ea96-2e5c-4e69-8d8b-a8882a476b19 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:34.101378 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d2ea96-2e5c-4e69-8d8b-a8882a476b19 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:34.103305 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 258/523] 10.222.0.71 () {56 vars in 1137 bytes} [Mon Jul 4 09:06:33 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:34.104133 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f767d5-a39f-49f6-87ce-f0a37076e1f7 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:34.121967 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 266/524] 10.222.0.71 () {60 vars in 1132 bytes} [Mon Jul 4 09:06:33 2022] POST /identity/v3/users => generated 247 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:06:34.122787 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d2ea96-2e5c-4e69-8d8b-a8882a476b19 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:34.218055 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f767d5-a39f-49f6-87ce-f0a37076e1f7 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:34.225906 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:34.253235 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d2ea96-2e5c-4e69-8d8b-a8882a476b19 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:34.256428 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dfa238-7700-41fc-906a-298f3be046fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:34.257141 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dfa238-7700-41fc-906a-298f3be046fb None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:34.257427 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dfa238-7700-41fc-906a-298f3be046fb None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:34.257706 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dfa238-7700-41fc-906a-298f3be046fb None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:34.259241 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dfa238-7700-41fc-906a-298f3be046fb None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:34.259766 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dfa238-7700-41fc-906a-298f3be046fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:34.261290 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dfa238-7700-41fc-906a-298f3be046fb None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:34.280997 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 259/525] 10.222.0.71 () {56 vars in 1139 bytes} [Mon Jul 4 09:06:34 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:34.281351 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dfa238-7700-41fc-906a-298f3be046fb None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:34.294622 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:34.300345 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335451ad-9e3e-4ede-b363-e13193e81ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:34.301448 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335451ad-9e3e-4ede-b363-e13193e81ca1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:34.302367 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335451ad-9e3e-4ede-b363-e13193e81ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:34.302367 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335451ad-9e3e-4ede-b363-e13193e81ca1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:34.303282 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335451ad-9e3e-4ede-b363-e13193e81ca1 None admin] RBAC: Adding query filter params (role.id=5752a9bdfe4b482ab37836eba7bd536b, scope.project.id=4076e316243946e980ce47386b18b28f, user.id=15039da209ec445d93d49e2bd6d854eb) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:34.303399 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335451ad-9e3e-4ede-b363-e13193e81ca1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:34.305693 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335451ad-9e3e-4ede-b363-e13193e81ca1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:34.312915 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 267/526] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 4 09:06:34 2022] GET /identity/v3/role_assignments?user.id=15039da209ec445d93d49e2bd6d854eb&scope.project.id=4076e316243946e980ce47386b18b28f&role.id=5752a9bdfe4b482ab37836eba7bd536b => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:34.313478 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335451ad-9e3e-4ede-b363-e13193e81ca1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:34.423545 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dfa238-7700-41fc-906a-298f3be046fb None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:34.444721 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335451ad-9e3e-4ede-b363-e13193e81ca1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:34.470734 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41cb0ee-0673-4571-9359-cd28225be91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:34.471549 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41cb0ee-0673-4571-9359-cd28225be91e None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:34.472002 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41cb0ee-0673-4571-9359-cd28225be91e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:34.472286 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41cb0ee-0673-4571-9359-cd28225be91e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:34.482174 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41cb0ee-0673-4571-9359-cd28225be91e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:34.483654 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41cb0ee-0673-4571-9359-cd28225be91e None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:34.484566 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfa1af0-9892-4728-958f-e7a4d59a1769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:34.485377 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfa1af0-9892-4728-958f-e7a4d59a1769 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:34.485775 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfa1af0-9892-4728-958f-e7a4d59a1769 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:34.486069 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfa1af0-9892-4728-958f-e7a4d59a1769 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:34.503095 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-b41cb0ee-0673-4571-9359-cd28225be91e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:06:34.503095 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:34.503095 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:34.503095 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:34.503095 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:34.503095 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:34.503095 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:34.503095 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:34.503095 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:34.503095 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:34.503095 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:34.503095 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:34.503095 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:06:34.503095 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:06:34.503095 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:06:34.504025 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:06:34.504025 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:34.504025 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:34.504025 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:34.504025 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:34.504025 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:34.504025 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:34.504025 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:34.504025 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:34.504025 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:34.504025 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:34.504025 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:34.504025 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:34.504025 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:34.504025 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:34.504025 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:34.504025 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:34.505104 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:06:34.505104 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:06:34.505104 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:06:34.505104 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:06:34.505104 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:06:34.505104 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:06:34.505104 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:06:34.505104 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:34.505813 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 260/527] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 4 09:06:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:34.506428 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41cb0ee-0673-4571-9359-cd28225be91e None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:34.507134 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:34.566649 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfa1af0-9892-4728-958f-e7a4d59a1769 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:34.568034 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfa1af0-9892-4728-958f-e7a4d59a1769 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:34.605462 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:34.650152 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41cb0ee-0673-4571-9359-cd28225be91e None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:34.679641 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 268/528] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:06:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:06:34.681068 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfa1af0-9892-4728-958f-e7a4d59a1769 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:34.697179 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8734688-02f8-4a58-bb73-f255ac497b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:34.698021 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8734688-02f8-4a58-bb73-f255ac497b78 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:34.698374 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8734688-02f8-4a58-bb73-f255ac497b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:34.698713 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8734688-02f8-4a58-bb73-f255ac497b78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:34.747828 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:34.781076 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8734688-02f8-4a58-bb73-f255ac497b78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:34.785067 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8734688-02f8-4a58-bb73-f255ac497b78 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:34.821206 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfa1af0-9892-4728-958f-e7a4d59a1769 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:34.863717 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eefc93a-c29c-4d96-a1ac-039f23ab2b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:34.864711 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eefc93a-c29c-4d96-a1ac-039f23ab2b10 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:34.865406 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eefc93a-c29c-4d96-a1ac-039f23ab2b10 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:34.866051 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eefc93a-c29c-4d96-a1ac-039f23ab2b10 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:34.866949 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eefc93a-c29c-4d96-a1ac-039f23ab2b10 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:34.867409 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eefc93a-c29c-4d96-a1ac-039f23ab2b10 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:34.869087 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eefc93a-c29c-4d96-a1ac-039f23ab2b10 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:34.869417 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:34.892536 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 269/529] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 4 09:06:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:34.893648 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eefc93a-c29c-4d96-a1ac-039f23ab2b10 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:34.915921 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 261/530] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:06:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:06:34.916946 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8734688-02f8-4a58-bb73-f255ac497b78 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:35.003533 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eefc93a-c29c-4d96-a1ac-039f23ab2b10 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:35.006020 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:35.025802 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8734688-02f8-4a58-bb73-f255ac497b78 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:35.041906 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbde2abe-0ac9-496c-8bbe-0fe4e0c12424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:35.042709 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde2abe-0ac9-496c-8bbe-0fe4e0c12424 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:35.042987 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde2abe-0ac9-496c-8bbe-0fe4e0c12424 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:35.043259 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde2abe-0ac9-496c-8bbe-0fe4e0c12424 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:35.055403 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbde2abe-0ac9-496c-8bbe-0fe4e0c12424 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:35.056891 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbde2abe-0ac9-496c-8bbe-0fe4e0c12424 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:35.062706 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c26c079-6241-4fcf-8f55-883a53f03c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:35.063464 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c26c079-6241-4fcf-8f55-883a53f03c9f None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:35.063755 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c26c079-6241-4fcf-8f55-883a53f03c9f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:35.064035 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c26c079-6241-4fcf-8f55-883a53f03c9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:35.064973 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c26c079-6241-4fcf-8f55-883a53f03c9f None admin] RBAC: Adding query filter params (role.id=5752a9bdfe4b482ab37836eba7bd536b, scope.project.id=4076e316243946e980ce47386b18b28f, user.id=192724527d774817bc4581474ac848dd) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:35.065319 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c26c079-6241-4fcf-8f55-883a53f03c9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:35.066745 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c26c079-6241-4fcf-8f55-883a53f03c9f None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:35.073784 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 262/531] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 4 09:06:34 2022] GET /identity/v3/role_assignments?user.id=192724527d774817bc4581474ac848dd&scope.project.id=4076e316243946e980ce47386b18b28f&role.id=5752a9bdfe4b482ab37836eba7bd536b => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:35.074532 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c26c079-6241-4fcf-8f55-883a53f03c9f None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:35.075509 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-cbde2abe-0ac9-496c-8bbe-0fe4e0c12424 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:35.075509 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:35.075509 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:35.075509 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:35.075509 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:35.075509 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:35.075509 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:35.075509 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:35.075509 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:35.075509 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:35.075509 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:35.075509 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:35.075509 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:06:35.075509 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:06:35.075509 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:06:35.077531 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:06:35.077531 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:35.077531 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:35.077531 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:35.077531 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:35.077531 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:35.077531 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:35.077531 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:35.077531 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:35.077531 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:35.077531 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:35.077531 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:35.077531 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:35.077531 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:35.077531 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:35.077531 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:35.077531 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:35.079784 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:06:35.079784 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:06:35.079784 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:35.079784 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:35.089180 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 270/532] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:06:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:35.089180 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbde2abe-0ac9-496c-8bbe-0fe4e0c12424 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:35.109775 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:35.226254 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c26c079-6241-4fcf-8f55-883a53f03c9f None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:35.255292 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbde2abe-0ac9-496c-8bbe-0fe4e0c12424 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:35.286195 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adccdc7f-3953-4313-a4e0-7b547fa0ee9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:35.287696 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adccdc7f-3953-4313-a4e0-7b547fa0ee9d None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:35.288185 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adccdc7f-3953-4313-a4e0-7b547fa0ee9d None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:35.288676 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adccdc7f-3953-4313-a4e0-7b547fa0ee9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:35.296335 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee32137-8a1d-4968-b8fc-9f56896b7f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:35.297004 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee32137-8a1d-4968-b8fc-9f56896b7f2f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:35.297332 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee32137-8a1d-4968-b8fc-9f56896b7f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:35.297611 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee32137-8a1d-4968-b8fc-9f56896b7f2f None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:35.298344 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee32137-8a1d-4968-b8fc-9f56896b7f2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:35.298654 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee32137-8a1d-4968-b8fc-9f56896b7f2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:35.300504 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee32137-8a1d-4968-b8fc-9f56896b7f2f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:35.304191 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adccdc7f-3953-4313-a4e0-7b547fa0ee9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:35.306006 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adccdc7f-3953-4313-a4e0-7b547fa0ee9d None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:35.315788 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 271/533] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:06:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:35.321615 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee32137-8a1d-4968-b8fc-9f56896b7f2f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:35.328043 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-adccdc7f-3953-4313-a4e0-7b547fa0ee9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:35.328043 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:35.328043 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:35.328043 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:35.328043 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:35.328043 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:35.328043 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:35.328043 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:35.328043 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:35.328043 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:35.328043 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:35.328043 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:35.328043 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:06:35.328043 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:06:35.328043 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:06:35.328894 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:06:35.328894 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:35.328894 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:35.328894 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:35.328894 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:35.328894 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:35.328894 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:35.328894 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:35.328894 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:35.328894 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:35.328894 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:35.328894 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:35.328894 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:35.328894 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:35.328894 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:35.328894 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:35.328894 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:35.329823 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:06:35.329823 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:06:35.329823 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:35.329823 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:35.336867 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 263/534] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:06:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:35.338737 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adccdc7f-3953-4313-a4e0-7b547fa0ee9d None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:35.361439 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:35.467508 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adccdc7f-3953-4313-a4e0-7b547fa0ee9d None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:35.497093 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:35.517796 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a46b5a-7322-4440-a001-17ea051cbe96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:35.518762 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a46b5a-7322-4440-a001-17ea051cbe96 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:35.519217 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a46b5a-7322-4440-a001-17ea051cbe96 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:35.521516 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a46b5a-7322-4440-a001-17ea051cbe96 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:35.522403 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a46b5a-7322-4440-a001-17ea051cbe96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:35.522863 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a46b5a-7322-4440-a001-17ea051cbe96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:35.525061 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a46b5a-7322-4440-a001-17ea051cbe96 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:35.550466 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 264/535] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:06:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:35.556836 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a46b5a-7322-4440-a001-17ea051cbe96 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:35.574628 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee32137-8a1d-4968-b8fc-9f56896b7f2f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:35.620432 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d4d0f4-aba6-4161-95f3-4de9f8277bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:35.621333 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d4d0f4-aba6-4161-95f3-4de9f8277bc6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:35.621924 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d4d0f4-aba6-4161-95f3-4de9f8277bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:35.622204 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d4d0f4-aba6-4161-95f3-4de9f8277bc6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:35.631187 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d4d0f4-aba6-4161-95f3-4de9f8277bc6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:35.632624 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d4d0f4-aba6-4161-95f3-4de9f8277bc6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:35.666758 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-39d4d0f4-aba6-4161-95f3-4de9f8277bc6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:06:35.666758 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:35.666758 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:35.666758 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:35.666758 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:35.666758 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:35.666758 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:35.666758 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:35.666758 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:35.666758 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:35.666758 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:35.666758 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:35.666758 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:06:35.666758 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:06:35.666758 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:06:35.667684 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:06:35.667684 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:35.667684 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:35.667684 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:35.667684 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:35.667684 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:35.667684 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:35.667684 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:35.667684 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:35.667684 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:35.667684 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:35.667684 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:35.667684 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:35.667684 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:35.667684 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:35.667684 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:35.667684 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:35.668543 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:06:35.668543 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:06:35.668543 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:06:35.668543 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:06:35.668543 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:06:35.668543 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:06:35.668543 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:06:35.668543 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:35.668543 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 272/536] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 4 09:06:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:35.674001 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d4d0f4-aba6-4161-95f3-4de9f8277bc6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:35.736344 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a46b5a-7322-4440-a001-17ea051cbe96 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:35.774385 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387bbf56-6d4a-4dbd-9800-4fd6e490709a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:35.775534 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387bbf56-6d4a-4dbd-9800-4fd6e490709a None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:35.775817 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387bbf56-6d4a-4dbd-9800-4fd6e490709a None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:35.776093 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387bbf56-6d4a-4dbd-9800-4fd6e490709a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:35.792517 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387bbf56-6d4a-4dbd-9800-4fd6e490709a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:35.794004 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387bbf56-6d4a-4dbd-9800-4fd6e490709a None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:35.825319 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-387bbf56-6d4a-4dbd-9800-4fd6e490709a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:06:35.825319 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:35.825319 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:35.825319 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:35.825319 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:35.825319 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:35.825319 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:35.825319 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:35.825319 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:35.825319 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:35.825319 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:35.825319 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:35.825319 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:06:35.825319 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:06:35.825319 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:06:35.826516 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:06:35.826516 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:35.826516 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:35.826516 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:35.826516 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:35.826516 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:35.826516 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:35.826516 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:35.826516 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:35.826516 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:35.826516 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:35.826516 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:35.826516 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:35.826516 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:35.826516 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:35.826516 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:35.826516 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:35.827598 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:06:35.827598 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:06:35.827598 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:06:35.827598 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:06:35.827598 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:06:35.827598 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:06:35.827598 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:06:35.827598 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:35.829166 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 265/537] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 4 09:06:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:06:35.834473 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387bbf56-6d4a-4dbd-9800-4fd6e490709a None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:35.839137 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:35.842435 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:35.895118 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d4d0f4-aba6-4161-95f3-4de9f8277bc6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:35.946304 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387bbf56-6d4a-4dbd-9800-4fd6e490709a None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:35.980556 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcee7367-c3d1-41df-a0fb-9302034b043e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:35.981255 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee7367-c3d1-41df-a0fb-9302034b043e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:35.981540 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee7367-c3d1-41df-a0fb-9302034b043e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:35.981818 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee7367-c3d1-41df-a0fb-9302034b043e None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:35.982610 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcee7367-c3d1-41df-a0fb-9302034b043e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:35.982921 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcee7367-c3d1-41df-a0fb-9302034b043e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:35.984346 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcee7367-c3d1-41df-a0fb-9302034b043e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:35.988438 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee14270-f928-49c7-bb20-752b20b7d142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:35.989190 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee14270-f928-49c7-bb20-752b20b7d142 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:35.989473 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee14270-f928-49c7-bb20-752b20b7d142 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:35.989745 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee14270-f928-49c7-bb20-752b20b7d142 None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:35.990595 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee14270-f928-49c7-bb20-752b20b7d142 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:35.990904 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee14270-f928-49c7-bb20-752b20b7d142 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:35.992329 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee14270-f928-49c7-bb20-752b20b7d142 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:36.012453 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 273/538] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 4 09:06:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:36.016271 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 266/539] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 4 09:06:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:36.022072 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcee7367-c3d1-41df-a0fb-9302034b043e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:36.022558 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee14270-f928-49c7-bb20-752b20b7d142 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:36.119610 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:36.165922 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee14270-f928-49c7-bb20-752b20b7d142 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:36.226508 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcee7367-c3d1-41df-a0fb-9302034b043e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:36.230837 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b319dd0-603e-491e-b346-e7b80a9b46db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:36.231504 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b319dd0-603e-491e-b346-e7b80a9b46db None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:36.231791 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b319dd0-603e-491e-b346-e7b80a9b46db None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:36.232078 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b319dd0-603e-491e-b346-e7b80a9b46db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:36.232923 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b319dd0-603e-491e-b346-e7b80a9b46db None admin] RBAC: Adding query filter params (role.id=5752a9bdfe4b482ab37836eba7bd536b, scope.project.id=4076e316243946e980ce47386b18b28f, user.id=15818e58661646f381da0973d4c2ff64) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:36.233273 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b319dd0-603e-491e-b346-e7b80a9b46db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:36.234843 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b319dd0-603e-491e-b346-e7b80a9b46db None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:36.245180 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 267/540] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 4 09:06:36 2022] GET /identity/v3/role_assignments?user.id=15818e58661646f381da0973d4c2ff64&scope.project.id=4076e316243946e980ce47386b18b28f&role.id=5752a9bdfe4b482ab37836eba7bd536b => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:36.293451 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cce0fe-a88c-44d1-a050-faf003738351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:36.294144 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cce0fe-a88c-44d1-a050-faf003738351 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:36.294428 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cce0fe-a88c-44d1-a050-faf003738351 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:36.294703 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cce0fe-a88c-44d1-a050-faf003738351 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:36.295570 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cce0fe-a88c-44d1-a050-faf003738351 None admin] RBAC: Adding query filter params (role.id=5752a9bdfe4b482ab37836eba7bd536b, scope.project.id=4076e316243946e980ce47386b18b28f, user.id=f6e50a3715a14714b3951d36338dde42) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:36.295883 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cce0fe-a88c-44d1-a050-faf003738351 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:36.297410 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cce0fe-a88c-44d1-a050-faf003738351 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:36.300952 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:36.309029 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 274/541] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 4 09:06:36 2022] GET /identity/v3/role_assignments?user.id=f6e50a3715a14714b3951d36338dde42&scope.project.id=4076e316243946e980ce47386b18b28f&role.id=5752a9bdfe4b482ab37836eba7bd536b => generated 259 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:36.459543 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c4975-82e9-40c5-ac64-49ebe3125765 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:36.460073 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c4975-82e9-40c5-ac64-49ebe3125765 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:36.460157 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c4975-82e9-40c5-ac64-49ebe3125765 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:36.461691 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 268/542] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:36.467697 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ecd85-5b7d-4bab-914b-dbac806ed3a8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:36.468302 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ecd85-5b7d-4bab-914b-dbac806ed3a8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:36.468785 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ecd85-5b7d-4bab-914b-dbac806ed3a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:36.487431 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-684ecd85-5b7d-4bab-914b-dbac806ed3a8 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:36.488989 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-684ecd85-5b7d-4bab-914b-dbac806ed3a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:36.581864 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 275/543] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:06:36.607150 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9c4975-82e9-40c5-ac64-49ebe3125765 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:36.739558 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9c4975-82e9-40c5-ac64-49ebe3125765 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:36.746004 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab04d5c-99b8-43c1-b940-1a6af6d8989d None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:36.746615 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab04d5c-99b8-43c1-b940-1a6af6d8989d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:36.747177 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab04d5c-99b8-43c1-b940-1a6af6d8989d None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:36.748789 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 276/544] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:06:36.753980 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c59c5-be87-4d26-9552-b786f71a107e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:36.754237 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c59c5-be87-4d26-9552-b786f71a107e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:36.754523 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c59c5-be87-4d26-9552-b786f71a107e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:36.776033 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:36.784638 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593ba0a3-00e8-4abc-8a33-9c2b89f64b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:36.786089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ba0a3-00e8-4abc-8a33-9c2b89f64b83 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:36.786619 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ba0a3-00e8-4abc-8a33-9c2b89f64b83 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:36.787090 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ba0a3-00e8-4abc-8a33-9c2b89f64b83 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:36.787319 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-132c59c5-be87-4d26-9552-b786f71a107e None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:36.788480 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-132c59c5-be87-4d26-9552-b786f71a107e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:36.799505 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593ba0a3-00e8-4abc-8a33-9c2b89f64b83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:06:36.810207 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-593ba0a3-00e8-4abc-8a33-9c2b89f64b83 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:06:36.810207 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:36.810207 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:36.810207 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:36.810207 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:36.810207 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:36.810207 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:36.810207 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:36.810207 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:36.810207 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:36.810207 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:36.810207 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:36.810207 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:06:36.810207 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:06:36.810207 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:06:36.810207 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:06:36.811129 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:06:36.811129 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:06:36.811129 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:06:36.811129 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:06:36.811129 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:36.811129 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:36.811129 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:36.811129 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:36.811129 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:36.811129 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:36.811129 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:36.811129 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:36.811129 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:36.811129 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:36.811129 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:36.811129 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:36.811940 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:36.811940 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:36.811940 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:36.811940 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:36.811940 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:06:36.811940 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:06:36.811940 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:06:36.811940 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:06:36.811940 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:06:36.811940 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:06:36.811940 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:06:36.811940 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:36.816568 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 269/545] 10.222.0.71 () {56 vars in 1102 bytes} [Mon Jul 4 09:06:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:36.822897 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593ba0a3-00e8-4abc-8a33-9c2b89f64b83 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:36.922609 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 277/546] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:06:36.945499 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c59c5-be87-4d26-9552-b786f71a107e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:37.009715 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:37.041485 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593ba0a3-00e8-4abc-8a33-9c2b89f64b83 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:37.053962 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c59c5-be87-4d26-9552-b786f71a107e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:37.088896 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c63b273-9db1-46fd-806c-37d8d7364552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:37.089610 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63b273-9db1-46fd-806c-37d8d7364552 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:37.089895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63b273-9db1-46fd-806c-37d8d7364552 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:37.090645 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63b273-9db1-46fd-806c-37d8d7364552 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:37.091685 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c63b273-9db1-46fd-806c-37d8d7364552 None admin] RBAC: Adding query filter params (name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:37.092001 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c63b273-9db1-46fd-806c-37d8d7364552 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:37.093452 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c63b273-9db1-46fd-806c-37d8d7364552 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:37.103082 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 270/547] 10.222.0.71 () {56 vars in 1103 bytes} [Mon Jul 4 09:06:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:06:37.110039 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c63b273-9db1-46fd-806c-37d8d7364552 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:37.134282 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555cc308-9d72-4777-a625-98beef9943c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:37.135093 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555cc308-9d72-4777-a625-98beef9943c3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:37.135375 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555cc308-9d72-4777-a625-98beef9943c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:37.135649 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555cc308-9d72-4777-a625-98beef9943c3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:37.156365 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555cc308-9d72-4777-a625-98beef9943c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:06:37.164005 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-555cc308-9d72-4777-a625-98beef9943c3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:06:37.164005 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:37.164005 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:37.164005 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:37.164005 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:37.164005 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:37.164005 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:37.164005 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:37.164005 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:37.164005 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:37.164005 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:37.164005 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:37.164005 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:06:37.164005 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:06:37.164005 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:06:37.164005 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:06:37.165364 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:06:37.165364 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:06:37.165364 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:06:37.165364 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:06:37.165364 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:37.165364 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:37.165364 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:37.165364 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:37.165364 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:37.165364 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:37.165364 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:37.165364 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:37.165364 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:37.165364 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:37.165364 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:37.165364 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:37.166398 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:37.166398 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:37.166398 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:37.166398 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:37.166398 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:06:37.166398 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:06:37.166398 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:06:37.166398 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:06:37.166398 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:06:37.166398 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:06:37.166398 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:06:37.166398 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:37.171552 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 278/548] 10.222.0.71 () {56 vars in 1102 bytes} [Mon Jul 4 09:06:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:06:37.177241 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555cc308-9d72-4777-a625-98beef9943c3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:37.206379 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:37.292264 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c63b273-9db1-46fd-806c-37d8d7364552 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:37.354755 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555cc308-9d72-4777-a625-98beef9943c3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:37.367009 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36dc479-09a6-4b7a-94ee-1722799c0981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:37.368360 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36dc479-09a6-4b7a-94ee-1722799c0981 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:37.368647 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36dc479-09a6-4b7a-94ee-1722799c0981 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:37.368916 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36dc479-09a6-4b7a-94ee-1722799c0981 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:37.381309 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:37.400142 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36dc479-09a6-4b7a-94ee-1722799c0981 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:37.401881 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36dc479-09a6-4b7a-94ee-1722799c0981 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:37.412164 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b82bb32-8b61-4c35-87cd-899d52f1b2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:37.413178 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b82bb32-8b61-4c35-87cd-899d52f1b2ea None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:37.413664 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b82bb32-8b61-4c35-87cd-899d52f1b2ea None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:37.414155 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b82bb32-8b61-4c35-87cd-899d52f1b2ea None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:37.415143 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b82bb32-8b61-4c35-87cd-899d52f1b2ea None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:37.415650 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b82bb32-8b61-4c35-87cd-899d52f1b2ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:37.417288 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b82bb32-8b61-4c35-87cd-899d52f1b2ea None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:37.434916 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-c36dc479-09a6-4b7a-94ee-1722799c0981 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:37.434916 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:37.434916 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:37.434916 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:37.434916 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:37.434916 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:37.434916 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:37.434916 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:37.434916 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:37.434916 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:37.434916 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:37.434916 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:37.434916 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:06:37.434916 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:06:37.434916 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:06:37.436321 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:06:37.436321 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:37.436321 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:37.436321 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:37.436321 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:37.436321 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:37.436321 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:37.436321 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:37.436321 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:37.436321 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:37.436321 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:37.436321 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:37.436321 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:37.436321 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:37.436321 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:37.436321 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:37.436321 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:37.437376 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:06:37.437376 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:06:37.437376 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:37.437376 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:37.439530 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 271/549] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:06:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:37.441838 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 279/550] 10.222.0.71 () {56 vars in 1103 bytes} [Mon Jul 4 09:06:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:37.446883 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b82bb32-8b61-4c35-87cd-899d52f1b2ea None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:37.448004 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36dc479-09a6-4b7a-94ee-1722799c0981 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:37.577862 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36dc479-09a6-4b7a-94ee-1722799c0981 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:37.589091 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:37.603181 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b82bb32-8b61-4c35-87cd-899d52f1b2ea None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:37.646309 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7edc0dd-5b35-46dd-a4f1-8a01781c8693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:37.647115 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7edc0dd-5b35-46dd-a4f1-8a01781c8693 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:37.647401 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7edc0dd-5b35-46dd-a4f1-8a01781c8693 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:37.647683 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7edc0dd-5b35-46dd-a4f1-8a01781c8693 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:37.651946 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ddc337-f68b-44cf-8a43-4ccac9518308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:37.652773 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ddc337-f68b-44cf-8a43-4ccac9518308 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:37.653228 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ddc337-f68b-44cf-8a43-4ccac9518308 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:37.653646 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ddc337-f68b-44cf-8a43-4ccac9518308 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:37.654520 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ddc337-f68b-44cf-8a43-4ccac9518308 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:37.655074 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ddc337-f68b-44cf-8a43-4ccac9518308 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:37.656572 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ddc337-f68b-44cf-8a43-4ccac9518308 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:37.659831 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7edc0dd-5b35-46dd-a4f1-8a01781c8693 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:37.661296 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7edc0dd-5b35-46dd-a4f1-8a01781c8693 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:37.675171 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 280/551] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:06:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:37.680925 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71d191-b9e7-475a-9d07-5cf11ca815ca None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:37.681468 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71d191-b9e7-475a-9d07-5cf11ca815ca None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:37.681908 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71d191-b9e7-475a-9d07-5cf11ca815ca None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:37.682787 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-c7edc0dd-5b35-46dd-a4f1-8a01781c8693 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:37.682787 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:37.682787 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:37.682787 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:37.682787 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:37.682787 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:37.682787 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:37.682787 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:37.682787 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:37.682787 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:37.682787 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:37.682787 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:37.682787 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:06:37.682787 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:06:37.682787 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:06:37.683915 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:06:37.683915 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:37.683915 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:37.683915 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:37.683915 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:37.683915 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:37.683915 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:37.683915 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:37.683915 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:37.683915 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:37.683915 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:37.683915 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:37.683915 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:37.683915 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:37.683915 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:37.683915 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:37.683915 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:37.684958 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:06:37.684958 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:06:37.684958 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:37.684958 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:37.684958 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 272/552] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:06:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:37.685264 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7edc0dd-5b35-46dd-a4f1-8a01781c8693 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:37.686428 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 281/553] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:37 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:37.689392 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca71d191-b9e7-475a-9d07-5cf11ca815ca None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:37.743450 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:37.801503 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca71d191-b9e7-475a-9d07-5cf11ca815ca None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:37.803251 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7edc0dd-5b35-46dd-a4f1-8a01781c8693 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:37.839079 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d14cb1-1def-407b-97ee-dc301f1357ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:37.839678 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d14cb1-1def-407b-97ee-dc301f1357ee None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:37.839961 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d14cb1-1def-407b-97ee-dc301f1357ee None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:37.840241 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d14cb1-1def-407b-97ee-dc301f1357ee None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:37.840980 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d14cb1-1def-407b-97ee-dc301f1357ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:37.841321 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d14cb1-1def-407b-97ee-dc301f1357ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:37.842735 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d14cb1-1def-407b-97ee-dc301f1357ee None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:37.845800 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a521736e-e3eb-44a0-959e-e54c2e109d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:37.847379 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a521736e-e3eb-44a0-959e-e54c2e109d12 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:37.848032 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a521736e-e3eb-44a0-959e-e54c2e109d12 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:37.848720 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a521736e-e3eb-44a0-959e-e54c2e109d12 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:37.855375 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 282/554] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:06:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:37.856722 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ad89e-921f-4869-9a45-c31a6952a725 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:37.857016 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ad89e-921f-4869-9a45-c31a6952a725 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:37.857334 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ad89e-921f-4869-9a45-c31a6952a725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:37.858944 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a521736e-e3eb-44a0-959e-e54c2e109d12 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:37.860595 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a521736e-e3eb-44a0-959e-e54c2e109d12 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:37.873806 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-397ad89e-921f-4869-9a45-c31a6952a725 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:37.875105 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-397ad89e-921f-4869-9a45-c31a6952a725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:37.877009 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-a521736e-e3eb-44a0-959e-e54c2e109d12 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 04 09:06:37.877009 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:37.877009 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:37.877009 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:37.877009 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:37.877009 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:37.877009 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:37.877009 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:37.877009 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:37.877009 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:37.877009 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:37.877009 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:37.877009 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:06:37.877009 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:06:37.877009 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:06:37.877009 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:06:37.878229 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:37.878229 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:37.878229 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:06:37.878229 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:06:37.878229 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:06:37.878229 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:06:37.878229 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:37.878229 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:37.878229 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:37.878229 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:37.878229 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:37.878229 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:37.878229 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:37.878229 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:37.878229 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:37.878229 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:37.878229 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:37.878229 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:37.879325 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:37.879325 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:37.879325 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:06:37.879325 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:06:37.879325 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:06:37.879325 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:06:37.879325 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:06:37.879325 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:06:37.879325 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 04 09:06:37.879325 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:37.886569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 273/555] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 4 09:06:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:37.887637 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a521736e-e3eb-44a0-959e-e54c2e109d12 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:37.906898 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:37.953149 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 283/556] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:06:37.954063 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397ad89e-921f-4869-9a45-c31a6952a725 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:38.001925 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:38.005193 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a521736e-e3eb-44a0-959e-e54c2e109d12 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:38.043275 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8d0f57-bea4-431e-bca0-fe3c3f52c94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:38.044512 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8d0f57-bea4-431e-bca0-fe3c3f52c94c None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:38.044950 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8d0f57-bea4-431e-bca0-fe3c3f52c94c None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:38.045401 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8d0f57-bea4-431e-bca0-fe3c3f52c94c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:38.054169 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8d0f57-bea4-431e-bca0-fe3c3f52c94c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:38.056566 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8d0f57-bea4-431e-bca0-fe3c3f52c94c None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:38.073254 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-ff8d0f57-bea4-431e-bca0-fe3c3f52c94c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 04 09:06:38.073254 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:38.073254 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:38.073254 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:38.073254 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:38.073254 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:38.073254 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:38.073254 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:38.073254 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:38.073254 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:38.073254 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:38.073254 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:38.073254 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:06:38.073254 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:06:38.073254 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:06:38.073254 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:06:38.074705 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:38.074705 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:38.074705 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:06:38.074705 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:06:38.074705 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:06:38.074705 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:06:38.074705 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:38.074705 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:38.074705 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:38.074705 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:38.074705 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:38.074705 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:38.074705 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:38.074705 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:38.074705 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:38.074705 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:38.074705 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:38.074705 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:38.076462 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:38.076462 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:38.076462 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:06:38.076462 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:06:38.076462 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:06:38.076462 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:06:38.076462 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:06:38.076462 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:06:38.076462 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 04 09:06:38.076462 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:38.077280 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397ad89e-921f-4869-9a45-c31a6952a725 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:38.079191 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 274/557] 10.222.0.71 () {56 vars in 1093 bytes} [Mon Jul 4 09:06:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:38.080231 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8d0f57-bea4-431e-bca0-fe3c3f52c94c None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:38.112932 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd9c177-48f3-41bc-985d-915cfa59654c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:38.113588 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd9c177-48f3-41bc-985d-915cfa59654c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:38.113848 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd9c177-48f3-41bc-985d-915cfa59654c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:38.114126 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd9c177-48f3-41bc-985d-915cfa59654c None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:38.115001 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd9c177-48f3-41bc-985d-915cfa59654c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:38.115255 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd9c177-48f3-41bc-985d-915cfa59654c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:38.116677 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd9c177-48f3-41bc-985d-915cfa59654c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:38.140078 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 284/558] 10.222.0.71 () {56 vars in 1135 bytes} [Mon Jul 4 09:06:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:38.140998 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd9c177-48f3-41bc-985d-915cfa59654c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:38.168315 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:38.197996 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8d0f57-bea4-431e-bca0-fe3c3f52c94c None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:38.236623 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:38.238399 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63c99a5-a88e-4fe0-9d05-39b6b2c7ae49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:38.239452 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c99a5-a88e-4fe0-9d05-39b6b2c7ae49 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:38.239889 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c99a5-a88e-4fe0-9d05-39b6b2c7ae49 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:38.240319 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c99a5-a88e-4fe0-9d05-39b6b2c7ae49 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:38.246416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd9c177-48f3-41bc-985d-915cfa59654c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:38.253414 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63c99a5-a88e-4fe0-9d05-39b6b2c7ae49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:06:38.262469 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-e63c99a5-a88e-4fe0-9d05-39b6b2c7ae49 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:06:38.262469 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:38.262469 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:38.262469 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:38.262469 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:38.262469 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:38.262469 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:38.262469 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:38.262469 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:38.262469 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:38.262469 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:38.262469 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:38.262469 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:06:38.262469 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:06:38.262469 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:06:38.262469 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:06:38.263544 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:06:38.263544 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:06:38.263544 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:06:38.263544 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:06:38.263544 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:38.263544 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:38.263544 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:38.263544 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:38.263544 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:38.263544 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:38.263544 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:38.263544 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:38.263544 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:38.263544 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:38.263544 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:38.263544 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:38.264423 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:38.264423 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:38.264423 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:38.264423 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:38.264423 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:06:38.264423 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:06:38.264423 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:06:38.264423 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:06:38.264423 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:06:38.264423 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:06:38.264423 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:06:38.264423 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:38.267193 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 275/559] 10.222.0.71 () {56 vars in 1102 bytes} [Mon Jul 4 09:06:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:38.268165 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63c99a5-a88e-4fe0-9d05-39b6b2c7ae49 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:38.288476 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d87b87-ad93-480d-ac5d-739675073795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:38.289349 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d87b87-ad93-480d-ac5d-739675073795 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:38.290276 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d87b87-ad93-480d-ac5d-739675073795 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:38.290626 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d87b87-ad93-480d-ac5d-739675073795 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:38.291536 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d87b87-ad93-480d-ac5d-739675073795 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:38.291914 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d87b87-ad93-480d-ac5d-739675073795 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:38.293475 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d87b87-ad93-480d-ac5d-739675073795 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:38.311109 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 285/560] 10.222.0.71 () {56 vars in 1133 bytes} [Mon Jul 4 09:06:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:38.312111 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d87b87-ad93-480d-ac5d-739675073795 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:38.395736 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63c99a5-a88e-4fe0-9d05-39b6b2c7ae49 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:38.414314 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d87b87-ad93-480d-ac5d-739675073795 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:38.447178 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7a2c8-c32b-4a8e-8dbe-2327071ac7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:38.448273 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7a2c8-c32b-4a8e-8dbe-2327071ac7ab None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:38.448748 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7a2c8-c32b-4a8e-8dbe-2327071ac7ab None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:38.449266 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7a2c8-c32b-4a8e-8dbe-2327071ac7ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:38.450197 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9513f7-f4ba-4f5d-8e2a-bb8fbf2037c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:38.450954 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9513f7-f4ba-4f5d-8e2a-bb8fbf2037c7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:38.451321 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9513f7-f4ba-4f5d-8e2a-bb8fbf2037c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:38.451667 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9513f7-f4ba-4f5d-8e2a-bb8fbf2037c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:38.452484 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9513f7-f4ba-4f5d-8e2a-bb8fbf2037c7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:38.452858 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9513f7-f4ba-4f5d-8e2a-bb8fbf2037c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:38.453612 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:38.454391 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9513f7-f4ba-4f5d-8e2a-bb8fbf2037c7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:38.464083 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 286/561] 10.222.0.71 () {56 vars in 1103 bytes} [Mon Jul 4 09:06:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:38.469766 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9513f7-f4ba-4f5d-8e2a-bb8fbf2037c7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:38.484576 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:38.529054 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e7a2c8-c32b-4a8e-8dbe-2327071ac7ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:38.530938 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e7a2c8-c32b-4a8e-8dbe-2327071ac7ab None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:38.594265 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9513f7-f4ba-4f5d-8e2a-bb8fbf2037c7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:38.632131 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6290f840-ef6b-4dbf-832f-5bcd2eadb9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:38.633103 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290f840-ef6b-4dbf-832f-5bcd2eadb9c6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:38.633376 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290f840-ef6b-4dbf-832f-5bcd2eadb9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:38.633961 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290f840-ef6b-4dbf-832f-5bcd2eadb9c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:38.637460 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 276/562] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:06:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:06:38.641940 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7a2c8-c32b-4a8e-8dbe-2327071ac7ab None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:38.645955 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6290f840-ef6b-4dbf-832f-5bcd2eadb9c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:38.647379 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6290f840-ef6b-4dbf-832f-5bcd2eadb9c6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:38.666288 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-6290f840-ef6b-4dbf-832f-5bcd2eadb9c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:38.666288 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:38.666288 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:38.666288 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:38.666288 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:38.666288 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:38.666288 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:38.666288 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:38.666288 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:38.666288 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:38.666288 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:38.666288 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:38.666288 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:06:38.666288 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:06:38.666288 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:06:38.667306 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:06:38.667306 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:38.667306 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:38.667306 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:38.667306 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:38.667306 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:38.667306 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:38.667306 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:38.667306 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:38.667306 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:38.667306 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:38.667306 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:38.667306 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:38.667306 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:38.667306 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:38.667306 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:38.667306 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:38.668258 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:06:38.668258 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:06:38.668258 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:38.668258 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:38.671984 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 287/563] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:06:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:38.673319 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e244f10b-a981-4bca-b0af-fefcdab6c46d None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:38.673611 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e244f10b-a981-4bca-b0af-fefcdab6c46d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:38.673893 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e244f10b-a981-4bca-b0af-fefcdab6c46d None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:38.675176 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 288/564] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:38.675865 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e244f10b-a981-4bca-b0af-fefcdab6c46d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:38.684334 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:38.741931 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:38.761368 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7a2c8-c32b-4a8e-8dbe-2327071ac7ab None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:38.787863 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e244f10b-a981-4bca-b0af-fefcdab6c46d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:38.799737 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5ecdfd-3118-40c7-b9c0-8c9d7ae99db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:38.800537 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ecdfd-3118-40c7-b9c0-8c9d7ae99db7 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:38.800876 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ecdfd-3118-40c7-b9c0-8c9d7ae99db7 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:38.801218 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ecdfd-3118-40c7-b9c0-8c9d7ae99db7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:38.811639 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5ecdfd-3118-40c7-b9c0-8c9d7ae99db7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:38.813120 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5ecdfd-3118-40c7-b9c0-8c9d7ae99db7 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:38.824891 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c519e5d-a596-4e87-8557-4cc1b2d5f7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:38.825705 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c519e5d-a596-4e87-8557-4cc1b2d5f7c7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:38.825988 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c519e5d-a596-4e87-8557-4cc1b2d5f7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:38.826262 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c519e5d-a596-4e87-8557-4cc1b2d5f7c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:38.831066 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-8c5ecdfd-3118-40c7-b9c0-8c9d7ae99db7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:38.831066 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:38.831066 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:38.831066 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:38.831066 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:38.831066 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:38.831066 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:38.831066 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:38.831066 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:38.831066 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:38.831066 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:38.831066 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:38.831066 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:06:38.831066 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:06:38.831066 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:06:38.832119 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:06:38.832119 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:38.832119 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:38.832119 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:38.832119 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:38.832119 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:38.832119 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:38.832119 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:38.832119 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:38.832119 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:38.832119 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:38.832119 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:38.832119 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:38.832119 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:38.832119 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:38.832119 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:38.832119 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:38.833415 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:06:38.833415 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:06:38.833415 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:38.833415 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:38.837623 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 277/565] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:06:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:38.838455 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5ecdfd-3118-40c7-b9c0-8c9d7ae99db7 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:38.838828 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c519e5d-a596-4e87-8557-4cc1b2d5f7c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:38.840284 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c519e5d-a596-4e87-8557-4cc1b2d5f7c7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:38.858727 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-7c519e5d-a596-4e87-8557-4cc1b2d5f7c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:38.858727 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:38.858727 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:38.858727 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:38.858727 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:38.858727 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:38.858727 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:38.858727 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:38.858727 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:38.858727 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:38.858727 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:38.858727 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:38.858727 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:06:38.858727 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:06:38.858727 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:06:38.859655 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:06:38.859655 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:38.859655 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:38.859655 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:38.859655 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:38.859655 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:38.859655 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:38.859655 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:38.859655 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:38.859655 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:38.859655 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:38.859655 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:38.859655 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:38.859655 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:38.859655 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:38.859655 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:38.859655 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:38.860579 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:06:38.860579 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:06:38.860579 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:38.860579 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:38.860579 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 289/566] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:06:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:38.861122 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4cf4a7-de7b-4205-bcb8-fb4440c1a399 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:38.861417 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4cf4a7-de7b-4205-bcb8-fb4440c1a399 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:38.861702 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4cf4a7-de7b-4205-bcb8-fb4440c1a399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:38.877937 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5e4cf4a7-de7b-4205-bcb8-fb4440c1a399 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:38.879139 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5e4cf4a7-de7b-4205-bcb8-fb4440c1a399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:38.925712 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:38.940999 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5ecdfd-3118-40c7-b9c0-8c9d7ae99db7 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:38.945053 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 290/567] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:06:38.946461 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab96cf-dea9-433a-9bbb-1f404a72f0be None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:38.946757 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab96cf-dea9-433a-9bbb-1f404a72f0be None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:38.947042 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab96cf-dea9-433a-9bbb-1f404a72f0be None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:38.952400 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 291/568] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:38.953094 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cab96cf-dea9-433a-9bbb-1f404a72f0be None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:38.977641 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a052c8-8f94-4bb5-a069-2abf933f439a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:38.978309 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a052c8-8f94-4bb5-a069-2abf933f439a None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:38.978923 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a052c8-8f94-4bb5-a069-2abf933f439a None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:38.978923 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a052c8-8f94-4bb5-a069-2abf933f439a None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:38.979570 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a052c8-8f94-4bb5-a069-2abf933f439a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:38.979882 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a052c8-8f94-4bb5-a069-2abf933f439a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:38.981368 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a052c8-8f94-4bb5-a069-2abf933f439a None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:38.997212 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 278/569] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:06:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:38.998534 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a052c8-8f94-4bb5-a069-2abf933f439a None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:39.012835 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:39.067287 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cab96cf-dea9-433a-9bbb-1f404a72f0be None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:39.098815 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a052c8-8f94-4bb5-a069-2abf933f439a None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:39.102301 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8515adfe-f58c-4df6-a403-39e2bfe1c83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:39.102969 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515adfe-f58c-4df6-a403-39e2bfe1c83b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:39.103249 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515adfe-f58c-4df6-a403-39e2bfe1c83b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:39.103521 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515adfe-f58c-4df6-a403-39e2bfe1c83b None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:39.104228 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8515adfe-f58c-4df6-a403-39e2bfe1c83b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:39.104532 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8515adfe-f58c-4df6-a403-39e2bfe1c83b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:39.105919 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8515adfe-f58c-4df6-a403-39e2bfe1c83b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:39.116736 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 292/570] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:06:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:39.118958 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0fb52f-860f-481b-b041-473280a55538 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:39.119251 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0fb52f-860f-481b-b041-473280a55538 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:39.119527 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0fb52f-860f-481b-b041-473280a55538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:39.135975 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-db0fb52f-860f-481b-b041-473280a55538 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:39.136355 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900f4b1c-5c51-40af-b6c9-25e8f024e934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:39.137026 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f4b1c-5c51-40af-b6c9-25e8f024e934 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:39.137188 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-db0fb52f-860f-481b-b041-473280a55538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:39.137348 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f4b1c-5c51-40af-b6c9-25e8f024e934 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:39.137631 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f4b1c-5c51-40af-b6c9-25e8f024e934 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:39.138355 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900f4b1c-5c51-40af-b6c9-25e8f024e934 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:39.138669 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900f4b1c-5c51-40af-b6c9-25e8f024e934 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:39.140047 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900f4b1c-5c51-40af-b6c9-25e8f024e934 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:39.151776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 279/571] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:06:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:39.152572 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900f4b1c-5c51-40af-b6c9-25e8f024e934 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:39.161957 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:39.208600 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 293/572] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:06:39.209492 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0fb52f-860f-481b-b041-473280a55538 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:39.251445 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:39.254809 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900f4b1c-5c51-40af-b6c9-25e8f024e934 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:39.288563 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5305f9df-4843-4b85-a4ba-09e44fb9a6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:39.289954 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5305f9df-4843-4b85-a4ba-09e44fb9a6c7 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:39.290240 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5305f9df-4843-4b85-a4ba-09e44fb9a6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:39.290521 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5305f9df-4843-4b85-a4ba-09e44fb9a6c7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:39.300591 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5305f9df-4843-4b85-a4ba-09e44fb9a6c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:06:39.309089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-5305f9df-4843-4b85-a4ba-09e44fb9a6c7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:06:39.309089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:39.309089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:39.309089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:39.309089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:39.309089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:39.309089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:39.309089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:39.309089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:39.309089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:39.309089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:39.309089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:39.309089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:06:39.309089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:06:39.309089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:06:39.309089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:06:39.310215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:06:39.310215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:06:39.310215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:06:39.310215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:06:39.310215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:39.310215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:39.310215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:39.310215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:39.310215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:39.310215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:39.310215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:39.310215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:39.310215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:39.310215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:39.310215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:39.310215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:39.311346 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:39.311346 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:39.311346 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:39.311346 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:39.311346 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:06:39.311346 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:06:39.311346 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:06:39.311346 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:06:39.311346 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:06:39.311346 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:06:39.311346 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:06:39.311346 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:39.311346 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 280/573] 10.222.0.71 () {56 vars in 1102 bytes} [Mon Jul 4 09:06:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:39.315831 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5305f9df-4843-4b85-a4ba-09e44fb9a6c7 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:39.319979 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0fb52f-860f-481b-b041-473280a55538 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:39.355347 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0f9ae2-56ab-441f-8670-826003f2937d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:39.356138 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0f9ae2-56ab-441f-8670-826003f2937d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:39.356416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0f9ae2-56ab-441f-8670-826003f2937d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:39.356703 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0f9ae2-56ab-441f-8670-826003f2937d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:39.363224 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0f9ae2-56ab-441f-8670-826003f2937d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:39.364647 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0f9ae2-56ab-441f-8670-826003f2937d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:39.378683 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-cb0f9ae2-56ab-441f-8670-826003f2937d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:06:39.378683 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:39.378683 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:39.378683 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:39.378683 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:39.378683 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:39.378683 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:39.378683 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:39.378683 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:39.378683 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:39.378683 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:39.378683 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:39.378683 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:06:39.378683 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:06:39.378683 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:06:39.379722 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:06:39.379722 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:39.379722 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:39.379722 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:39.379722 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:39.379722 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:39.379722 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:39.379722 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:39.379722 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:39.379722 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:39.379722 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:39.379722 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:39.379722 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:39.379722 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:39.379722 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:39.379722 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:39.379722 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:39.380633 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:06:39.380633 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:06:39.380633 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:06:39.380633 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:06:39.380633 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:06:39.380633 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:06:39.380633 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:06:39.380633 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:39.380633 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 294/574] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 4 09:06:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:39.381197 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0f9ae2-56ab-441f-8670-826003f2937d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:39.381904 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:39.420967 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5305f9df-4843-4b85-a4ba-09e44fb9a6c7 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:39.454215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b747e0d3-ee32-4333-9f18-e159c7393235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:39.455007 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b747e0d3-ee32-4333-9f18-e159c7393235 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:39.455291 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b747e0d3-ee32-4333-9f18-e159c7393235 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:39.455566 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b747e0d3-ee32-4333-9f18-e159c7393235 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:39.464353 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b747e0d3-ee32-4333-9f18-e159c7393235 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:39.465772 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b747e0d3-ee32-4333-9f18-e159c7393235 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:39.479323 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:39.480428 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-b747e0d3-ee32-4333-9f18-e159c7393235 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 04 09:06:39.480428 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:39.480428 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:39.480428 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:39.480428 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:39.480428 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:39.480428 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:39.480428 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:39.480428 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:39.480428 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:39.480428 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:39.480428 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:39.480428 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:06:39.480428 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:06:39.480428 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:06:39.480428 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:06:39.481383 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:39.481383 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:39.481383 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:06:39.481383 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:06:39.481383 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:06:39.481383 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:06:39.481383 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:39.481383 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:39.481383 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:39.481383 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:39.481383 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:39.481383 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:39.481383 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:39.481383 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:39.481383 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:39.481383 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:39.481383 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:39.481383 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:39.487290 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:39.487290 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:39.487290 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:06:39.487290 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:06:39.487290 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:06:39.487290 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:06:39.487290 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:06:39.487290 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:06:39.487290 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 04 09:06:39.487290 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:39.487290 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 281/575] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:06:39 2022] GET /identity/v3/users/glance => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:39.487290 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b747e0d3-ee32-4333-9f18-e159c7393235 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:39.488037 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0f9ae2-56ab-441f-8670-826003f2937d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:39.521657 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f28ad7-0ec4-4ae0-837f-ffcddd02f9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:39.522446 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f28ad7-0ec4-4ae0-837f-ffcddd02f9ed None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:39.522730 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f28ad7-0ec4-4ae0-837f-ffcddd02f9ed None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:39.523011 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f28ad7-0ec4-4ae0-837f-ffcddd02f9ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:39.530198 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f28ad7-0ec4-4ae0-837f-ffcddd02f9ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:39.531646 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f28ad7-0ec4-4ae0-837f-ffcddd02f9ed None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:39.546231 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-49f28ad7-0ec4-4ae0-837f-ffcddd02f9ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:06:39.546231 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:39.546231 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:39.546231 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:39.546231 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:39.546231 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:39.546231 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:39.546231 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:39.546231 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:39.546231 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:39.546231 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:39.546231 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:39.546231 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:06:39.546231 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:06:39.546231 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:06:39.547324 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:06:39.547324 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:39.547324 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:39.547324 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:39.547324 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:39.547324 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:39.547324 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:39.547324 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:39.547324 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:39.547324 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:39.547324 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:39.547324 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:39.547324 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:39.547324 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:39.547324 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:39.547324 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:39.547324 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:39.548242 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:06:39.548242 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:06:39.548242 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:06:39.548242 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:06:39.548242 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:06:39.548242 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:06:39.548242 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:06:39.548242 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:39.548242 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 295/576] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 4 09:06:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:39.548745 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f28ad7-0ec4-4ae0-837f-ffcddd02f9ed None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:39.588192 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b747e0d3-ee32-4333-9f18-e159c7393235 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:39.613658 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:39.622959 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56458e8-ad89-41ac-b0a9-0bcad825f2db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:39.623748 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56458e8-ad89-41ac-b0a9-0bcad825f2db None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:39.624032 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56458e8-ad89-41ac-b0a9-0bcad825f2db None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:39.624310 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56458e8-ad89-41ac-b0a9-0bcad825f2db None admin] PATH_INFO: `/v3/roles/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:39.635260 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56458e8-ad89-41ac-b0a9-0bcad825f2db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:06:39.642822 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-c56458e8-ad89-41ac-b0a9-0bcad825f2db None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:06:39.642822 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:39.642822 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:39.642822 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:39.642822 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:39.642822 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:39.642822 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:39.642822 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:39.642822 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:39.642822 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:39.642822 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:39.642822 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:39.642822 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:06:39.642822 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:06:39.642822 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:06:39.642822 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:06:39.644353 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:06:39.644353 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:06:39.644353 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:06:39.644353 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:06:39.644353 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:39.644353 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:39.644353 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:39.644353 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:39.644353 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:39.644353 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:39.644353 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:39.644353 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:39.644353 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:39.644353 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:39.644353 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:39.644353 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:39.645637 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:39.645637 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:39.645637 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:39.645637 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:39.645637 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:06:39.645637 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:06:39.645637 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:06:39.645637 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:06:39.645637 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:06:39.645637 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:06:39.645637 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:06:39.645637 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:39.645637 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 282/577] 10.222.0.71 () {56 vars in 1102 bytes} [Mon Jul 4 09:06:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:39.645637 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56458e8-ad89-41ac-b0a9-0bcad825f2db None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:39.651250 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f28ad7-0ec4-4ae0-837f-ffcddd02f9ed None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:39.685507 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2682dd91-2062-4307-8b36-d9c622e8969b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:39.686194 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2682dd91-2062-4307-8b36-d9c622e8969b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:39.686480 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2682dd91-2062-4307-8b36-d9c622e8969b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:39.686758 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2682dd91-2062-4307-8b36-d9c622e8969b None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:39.687499 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2682dd91-2062-4307-8b36-d9c622e8969b None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:39.687801 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2682dd91-2062-4307-8b36-d9c622e8969b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:39.689189 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2682dd91-2062-4307-8b36-d9c622e8969b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:39.697482 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 296/578] 10.222.0.71 () {56 vars in 1103 bytes} [Mon Jul 4 09:06:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:39.698237 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2682dd91-2062-4307-8b36-d9c622e8969b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:39.718799 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:39.745831 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56458e8-ad89-41ac-b0a9-0bcad825f2db None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:39.778938 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2878612a-878f-4aad-a7ce-afbb319480a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:39.779590 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2878612a-878f-4aad-a7ce-afbb319480a4 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:39.779872 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2878612a-878f-4aad-a7ce-afbb319480a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:39.780150 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2878612a-878f-4aad-a7ce-afbb319480a4 None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:39.780923 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2878612a-878f-4aad-a7ce-afbb319480a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:39.781261 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2878612a-878f-4aad-a7ce-afbb319480a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:39.782700 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2878612a-878f-4aad-a7ce-afbb319480a4 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:39.794996 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 283/579] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 4 09:06:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:39.795814 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2878612a-878f-4aad-a7ce-afbb319480a4 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:39.802395 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2682dd91-2062-4307-8b36-d9c622e8969b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:39.836653 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f25472-cfe6-4c82-8ec3-9d75d77742f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:39.837623 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f25472-cfe6-4c82-8ec3-9d75d77742f4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:39.837908 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f25472-cfe6-4c82-8ec3-9d75d77742f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:39.838188 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f25472-cfe6-4c82-8ec3-9d75d77742f4 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:39.839026 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f25472-cfe6-4c82-8ec3-9d75d77742f4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:39.839335 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f25472-cfe6-4c82-8ec3-9d75d77742f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:39.840762 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f25472-cfe6-4c82-8ec3-9d75d77742f4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:39.847357 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:39.858355 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 297/580] 10.222.0.71 () {56 vars in 1137 bytes} [Mon Jul 4 09:06:39 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:39.859168 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f25472-cfe6-4c82-8ec3-9d75d77742f4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:39.901968 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2878612a-878f-4aad-a7ce-afbb319480a4 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:39.932913 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:39.941778 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc17387f-cdb3-4a4d-a72e-e8f10a9e7e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:39.942436 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc17387f-cdb3-4a4d-a72e-e8f10a9e7e5c None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:39.942721 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc17387f-cdb3-4a4d-a72e-e8f10a9e7e5c None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:39.942998 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc17387f-cdb3-4a4d-a72e-e8f10a9e7e5c None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:39.943782 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc17387f-cdb3-4a4d-a72e-e8f10a9e7e5c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:39.944090 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc17387f-cdb3-4a4d-a72e-e8f10a9e7e5c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:39.946227 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc17387f-cdb3-4a4d-a72e-e8f10a9e7e5c None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:39.957891 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 284/581] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 4 09:06:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:39.958124 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f25472-cfe6-4c82-8ec3-9d75d77742f4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:39.958695 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc17387f-cdb3-4a4d-a72e-e8f10a9e7e5c None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:39.995188 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9e75b1-b63f-44d3-8170-5ea7b750aeaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:39.995783 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e75b1-b63f-44d3-8170-5ea7b750aeaf None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:39.996063 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e75b1-b63f-44d3-8170-5ea7b750aeaf None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:39.996337 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e75b1-b63f-44d3-8170-5ea7b750aeaf None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:39.997097 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9e75b1-b63f-44d3-8170-5ea7b750aeaf None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:39.997405 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9e75b1-b63f-44d3-8170-5ea7b750aeaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:39.998761 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9e75b1-b63f-44d3-8170-5ea7b750aeaf None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:40.006463 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 298/582] 10.222.0.71 () {56 vars in 1103 bytes} [Mon Jul 4 09:06:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:40.007205 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9e75b1-b63f-44d3-8170-5ea7b750aeaf None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:40.061754 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc17387f-cdb3-4a4d-a72e-e8f10a9e7e5c None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:40.068474 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:40.097021 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af50674-4d60-4aae-bca4-68da288efc2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:40.097839 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af50674-4d60-4aae-bca4-68da288efc2a None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:40.098123 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af50674-4d60-4aae-bca4-68da288efc2a None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:40.098399 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af50674-4d60-4aae-bca4-68da288efc2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:40.109468 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af50674-4d60-4aae-bca4-68da288efc2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:40.110927 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af50674-4d60-4aae-bca4-68da288efc2a None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:40.113173 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9e75b1-b63f-44d3-8170-5ea7b750aeaf None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:40.129060 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-6af50674-4d60-4aae-bca4-68da288efc2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:40.129060 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:40.129060 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:40.129060 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:40.129060 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:40.129060 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:40.129060 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:40.129060 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:40.129060 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:40.129060 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:40.129060 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:40.129060 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:40.129060 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:06:40.129060 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:06:40.129060 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:06:40.130120 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:06:40.130120 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:40.130120 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:40.130120 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:40.130120 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:40.130120 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:40.130120 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:40.130120 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:40.130120 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:40.130120 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:40.130120 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:40.130120 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:40.130120 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:40.130120 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:40.130120 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:40.130120 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:40.130120 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:40.131134 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:06:40.131134 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:06:40.131134 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:40.131134 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:40.131134 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 285/583] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:06:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:40.131134 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af50674-4d60-4aae-bca4-68da288efc2a None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:40.151556 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7234266-3329-455e-8b7f-15a7433ea562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:40.152433 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7234266-3329-455e-8b7f-15a7433ea562 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:40.152709 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7234266-3329-455e-8b7f-15a7433ea562 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:40.152980 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7234266-3329-455e-8b7f-15a7433ea562 None admin] PATH_INFO: `/v3/projects/4076e316243946e980ce47386b18b28f/users/15039da209ec445d93d49e2bd6d854eb/roles/5752a9bdfe4b482ab37836eba7bd536b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:40.156214 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:40.168337 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7234266-3329-455e-8b7f-15a7433ea562 None admin] RBAC: Authorizing `identity:create_grant(project_id=4076e316243946e980ce47386b18b28f, user_id=15039da209ec445d93d49e2bd6d854eb, role_id=5752a9bdfe4b482ab37836eba7bd536b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:40.169796 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7234266-3329-455e-8b7f-15a7433ea562 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:40.183611 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 299/584] 10.222.0.71 () {58 vars in 1439 bytes} [Mon Jul 4 09:06:40 2022] PUT /identity/v3/projects/4076e316243946e980ce47386b18b28f/users/15039da209ec445d93d49e2bd6d854eb/roles/5752a9bdfe4b482ab37836eba7bd536b => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:06:40.184441 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7234266-3329-455e-8b7f-15a7433ea562 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:40.257861 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af50674-4d60-4aae-bca4-68da288efc2a None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:40.296200 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678288d9-4be3-4c42-8ba9-496f6d90cb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:40.297021 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678288d9-4be3-4c42-8ba9-496f6d90cb8c None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:40.297330 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678288d9-4be3-4c42-8ba9-496f6d90cb8c None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:40.297607 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678288d9-4be3-4c42-8ba9-496f6d90cb8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:40.298852 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7234266-3329-455e-8b7f-15a7433ea562 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:40.309483 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678288d9-4be3-4c42-8ba9-496f6d90cb8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:40.310984 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678288d9-4be3-4c42-8ba9-496f6d90cb8c None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:40.329060 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-678288d9-4be3-4c42-8ba9-496f6d90cb8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:40.329060 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:40.329060 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:40.329060 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:40.329060 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:40.329060 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:40.329060 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:40.329060 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:40.329060 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:40.329060 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:40.329060 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:40.329060 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:40.329060 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:06:40.329060 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:06:40.329060 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:06:40.330037 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:06:40.330037 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:40.330037 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:40.330037 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:40.330037 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:40.330037 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:40.330037 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:40.330037 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:40.330037 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:40.330037 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:40.330037 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:40.330037 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:40.330037 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:40.330037 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:40.330037 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:40.330037 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:40.330037 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:40.330946 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:06:40.330946 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:06:40.330946 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:40.330946 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:40.330946 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 286/585] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:06:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:40.331223 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678288d9-4be3-4c42-8ba9-496f6d90cb8c None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:40.341216 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf9fcfe-6b12-4a7b-990c-2aee0599a0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:40.341895 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9fcfe-6b12-4a7b-990c-2aee0599a0a3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:40.342183 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9fcfe-6b12-4a7b-990c-2aee0599a0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:40.342461 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9fcfe-6b12-4a7b-990c-2aee0599a0a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:40.343306 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf9fcfe-6b12-4a7b-990c-2aee0599a0a3 None admin] RBAC: Adding query filter params (role.id=5752a9bdfe4b482ab37836eba7bd536b, scope.project.id=4076e316243946e980ce47386b18b28f, user.id=88b41b55769a4c449716c3590ed58225) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:40.343623 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf9fcfe-6b12-4a7b-990c-2aee0599a0a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:40.345119 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf9fcfe-6b12-4a7b-990c-2aee0599a0a3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:40.351036 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 300/586] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 4 09:06:40 2022] GET /identity/v3/role_assignments?user.id=88b41b55769a4c449716c3590ed58225&scope.project.id=4076e316243946e980ce47386b18b28f&role.id=5752a9bdfe4b482ab37836eba7bd536b => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:40.351176 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:40.352457 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf9fcfe-6b12-4a7b-990c-2aee0599a0a3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:40.420457 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:40.480833 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678288d9-4be3-4c42-8ba9-496f6d90cb8c None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:40.492047 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf9fcfe-6b12-4a7b-990c-2aee0599a0a3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:40.524065 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35465e5e-3db1-40ee-9947-101c9eb8df7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:40.525272 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35465e5e-3db1-40ee-9947-101c9eb8df7c None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:40.525841 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35465e5e-3db1-40ee-9947-101c9eb8df7c None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:40.526313 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35465e5e-3db1-40ee-9947-101c9eb8df7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:40.531202 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c94bd2-1e29-4960-b1b8-65194c6c272a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:40.531970 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c94bd2-1e29-4960-b1b8-65194c6c272a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:40.532323 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c94bd2-1e29-4960-b1b8-65194c6c272a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:40.532672 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c94bd2-1e29-4960-b1b8-65194c6c272a None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:40.533508 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c94bd2-1e29-4960-b1b8-65194c6c272a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:40.533884 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c94bd2-1e29-4960-b1b8-65194c6c272a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:40.535328 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c94bd2-1e29-4960-b1b8-65194c6c272a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:40.541474 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35465e5e-3db1-40ee-9947-101c9eb8df7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:40.542969 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35465e5e-3db1-40ee-9947-101c9eb8df7c None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:40.551403 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 301/587] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:06:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:40.552265 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c94bd2-1e29-4960-b1b8-65194c6c272a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:40.561560 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-35465e5e-3db1-40ee-9947-101c9eb8df7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:40.561560 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:40.561560 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:40.561560 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:40.561560 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:40.561560 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:40.561560 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:40.561560 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:40.561560 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:40.561560 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:40.561560 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:40.561560 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:40.561560 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:06:40.561560 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:06:40.561560 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:06:40.562499 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:06:40.562499 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:40.562499 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:40.562499 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:40.562499 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:40.562499 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:40.562499 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:40.562499 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:40.562499 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:40.562499 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:40.562499 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:40.562499 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:40.562499 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:40.562499 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:40.562499 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:40.562499 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:40.562499 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:40.564043 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:06:40.564043 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:06:40.564043 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:40.564043 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:40.564043 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 287/588] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:06:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:40.564043 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35465e5e-3db1-40ee-9947-101c9eb8df7c None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:40.628327 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:40.672957 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35465e5e-3db1-40ee-9947-101c9eb8df7c None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:40.673616 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c94bd2-1e29-4960-b1b8-65194c6c272a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:40.703824 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:40.715209 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfea0b2-499e-4a24-9035-1650fe0d339a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:40.716026 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfea0b2-499e-4a24-9035-1650fe0d339a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:40.716629 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfea0b2-499e-4a24-9035-1650fe0d339a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:40.716913 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfea0b2-499e-4a24-9035-1650fe0d339a None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:40.717942 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfea0b2-499e-4a24-9035-1650fe0d339a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:40.718254 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfea0b2-499e-4a24-9035-1650fe0d339a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:40.720356 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfea0b2-499e-4a24-9035-1650fe0d339a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:40.724777 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de94b00-751d-4ec1-b133-b4079f275647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:40.725693 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de94b00-751d-4ec1-b133-b4079f275647 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:40.726049 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de94b00-751d-4ec1-b133-b4079f275647 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:40.726397 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de94b00-751d-4ec1-b133-b4079f275647 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:40.733474 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 302/589] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:06:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:40.734187 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfea0b2-499e-4a24-9035-1650fe0d339a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:40.739913 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de94b00-751d-4ec1-b133-b4079f275647 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:40.741425 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de94b00-751d-4ec1-b133-b4079f275647 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:40.756619 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-5de94b00-751d-4ec1-b133-b4079f275647 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 04 09:06:40.756619 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:40.756619 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:40.756619 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:40.756619 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:40.756619 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:40.756619 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:40.756619 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:40.756619 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:40.756619 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:40.756619 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:40.756619 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:40.756619 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:06:40.756619 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:06:40.756619 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:06:40.756619 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:06:40.757654 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:40.757654 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:40.757654 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:06:40.757654 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:06:40.757654 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:06:40.757654 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:06:40.757654 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:40.757654 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:40.757654 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:40.757654 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:40.757654 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:40.757654 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:40.757654 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:40.757654 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:40.757654 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:40.757654 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:40.757654 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:40.757654 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:40.760278 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:40.760278 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:40.760278 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:06:40.760278 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:06:40.760278 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:06:40.760278 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:06:40.760278 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:06:40.760278 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:06:40.760278 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 04 09:06:40.760278 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:40.760278 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 288/590] 10.222.0.71 () {56 vars in 1102 bytes} [Mon Jul 4 09:06:40 2022] GET /identity/v3/users/neutron => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:40.760278 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de94b00-751d-4ec1-b133-b4079f275647 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:40.871059 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de94b00-751d-4ec1-b133-b4079f275647 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:40.884204 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfea0b2-499e-4a24-9035-1650fe0d339a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:40.896535 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:40.910383 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc71b7c-036a-4a6f-929f-6f480e9d082f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:40.911195 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc71b7c-036a-4a6f-929f-6f480e9d082f None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:40.911481 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc71b7c-036a-4a6f-929f-6f480e9d082f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:40.911757 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc71b7c-036a-4a6f-929f-6f480e9d082f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:40.918832 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc71b7c-036a-4a6f-929f-6f480e9d082f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:40.920287 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc71b7c-036a-4a6f-929f-6f480e9d082f None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:40.930672 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6b7382-48b8-4e66-946b-9b93c08f817f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:40.931755 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b7382-48b8-4e66-946b-9b93c08f817f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:40.932245 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b7382-48b8-4e66-946b-9b93c08f817f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:40.932721 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b7382-48b8-4e66-946b-9b93c08f817f None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:40.933694 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6b7382-48b8-4e66-946b-9b93c08f817f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:40.934219 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6b7382-48b8-4e66-946b-9b93c08f817f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:40.935306 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-cfc71b7c-036a-4a6f-929f-6f480e9d082f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:06:40.935306 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:40.935306 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:40.935306 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:40.935306 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:40.935306 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:40.935306 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:40.935306 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:40.935306 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:40.935306 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:40.935306 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:40.935306 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:40.935306 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:06:40.935306 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:06:40.935306 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:06:40.936247 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:06:40.936247 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:40.936247 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:40.936247 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:40.936247 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:40.936247 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:40.936247 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:40.936247 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:40.936247 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:40.936247 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:40.936247 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:40.936247 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:40.936247 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:40.936247 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:40.936247 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:40.936247 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:40.936247 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:40.937188 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:06:40.937188 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:06:40.937188 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:06:40.937188 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:06:40.937188 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:06:40.937188 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:06:40.937188 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:06:40.937188 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:40.938663 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6b7382-48b8-4e66-946b-9b93c08f817f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:40.941217 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 289/591] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 4 09:06:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:40.942418 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc71b7c-036a-4a6f-929f-6f480e9d082f None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:40.960400 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 303/592] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:06:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:40.960400 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6b7382-48b8-4e66-946b-9b93c08f817f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:40.992584 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:41.076318 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6b7382-48b8-4e66-946b-9b93c08f817f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:41.081670 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc71b7c-036a-4a6f-929f-6f480e9d082f None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:41.112581 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e2104e-51ed-4a10-82eb-d3c1759a4849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:41.113365 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e2104e-51ed-4a10-82eb-d3c1759a4849 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:41.113570 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e2104e-51ed-4a10-82eb-d3c1759a4849 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:41.113826 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e2104e-51ed-4a10-82eb-d3c1759a4849 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:41.114621 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e2104e-51ed-4a10-82eb-d3c1759a4849 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:41.115163 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e2104e-51ed-4a10-82eb-d3c1759a4849 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:41.116345 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e2104e-51ed-4a10-82eb-d3c1759a4849 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:41.128938 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef45ad0-316a-4bf9-86c5-af06a91a76a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:41.129950 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef45ad0-316a-4bf9-86c5-af06a91a76a8 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:41.130440 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef45ad0-316a-4bf9-86c5-af06a91a76a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:41.130938 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef45ad0-316a-4bf9-86c5-af06a91a76a8 None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:41.131980 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef45ad0-316a-4bf9-86c5-af06a91a76a8 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:41.132493 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef45ad0-316a-4bf9-86c5-af06a91a76a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:41.134859 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 304/593] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 4 09:06:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:41.134978 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef45ad0-316a-4bf9-86c5-af06a91a76a8 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:41.135734 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e2104e-51ed-4a10-82eb-d3c1759a4849 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:41.159420 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 290/594] 10.222.0.71 () {56 vars in 1139 bytes} [Mon Jul 4 09:06:40 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:41.160944 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef45ad0-316a-4bf9-86c5-af06a91a76a8 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:41.218495 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:41.246277 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:41.260656 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e2104e-51ed-4a10-82eb-d3c1759a4849 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:41.281662 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef45ad0-316a-4bf9-86c5-af06a91a76a8 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:41.306463 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59bb31f-83d0-491d-8cbc-00c931ad7d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:41.308242 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59bb31f-83d0-491d-8cbc-00c931ad7d15 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:41.308886 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59bb31f-83d0-491d-8cbc-00c931ad7d15 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:41.309502 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59bb31f-83d0-491d-8cbc-00c931ad7d15 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:41.318950 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b363ac-1c1b-4452-b919-e7cb996c3e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:41.319853 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b363ac-1c1b-4452-b919-e7cb996c3e27 None admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:41.320167 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b363ac-1c1b-4452-b919-e7cb996c3e27 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:41.321001 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b363ac-1c1b-4452-b919-e7cb996c3e27 None admin] PATH_INFO: `/v3/projects/4076e316243946e980ce47386b18b28f/users/192724527d774817bc4581474ac848dd/roles/5752a9bdfe4b482ab37836eba7bd536b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:41.321967 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59bb31f-83d0-491d-8cbc-00c931ad7d15 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:41.323904 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59bb31f-83d0-491d-8cbc-00c931ad7d15 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:41.336721 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b363ac-1c1b-4452-b919-e7cb996c3e27 None admin] RBAC: Authorizing `identity:create_grant(project_id=4076e316243946e980ce47386b18b28f, user_id=192724527d774817bc4581474ac848dd, role_id=5752a9bdfe4b482ab37836eba7bd536b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:41.338657 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b363ac-1c1b-4452-b919-e7cb996c3e27 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:41.342946 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-b59bb31f-83d0-491d-8cbc-00c931ad7d15 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 04 09:06:41.342946 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:41.342946 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:41.342946 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:41.342946 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:41.342946 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:41.342946 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:41.342946 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:41.342946 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:41.342946 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:41.342946 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:41.342946 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:41.342946 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:06:41.342946 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:06:41.342946 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:06:41.342946 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:06:41.343897 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:41.343897 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:41.343897 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:06:41.343897 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:06:41.343897 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:06:41.343897 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:06:41.343897 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:41.343897 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:41.343897 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:41.343897 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:41.343897 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:41.343897 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:41.343897 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:41.343897 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:41.343897 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:41.343897 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:41.343897 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:41.343897 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:41.344854 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:41.344854 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:41.344854 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:06:41.344854 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:06:41.344854 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:06:41.344854 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:06:41.344854 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:06:41.344854 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:06:41.344854 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 04 09:06:41.344854 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:41.352419 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 305/595] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:06:41 2022] GET /identity/v3/users/cinder => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:41.352419 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59bb31f-83d0-491d-8cbc-00c931ad7d15 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:41.360264 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 291/596] 10.222.0.71 () {58 vars in 1439 bytes} [Mon Jul 4 09:06:41 2022] PUT /identity/v3/projects/4076e316243946e980ce47386b18b28f/users/192724527d774817bc4581474ac848dd/roles/5752a9bdfe4b482ab37836eba7bd536b => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:06:41.365290 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b363ac-1c1b-4452-b919-e7cb996c3e27 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:41.533472 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:41.542753 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b363ac-1c1b-4452-b919-e7cb996c3e27 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:41.583239 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59bb31f-83d0-491d-8cbc-00c931ad7d15 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:41.584558 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250cb95a-de19-4ba4-8e23-cfa0f06e38ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:41.585480 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250cb95a-de19-4ba4-8e23-cfa0f06e38ce None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:41.585770 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250cb95a-de19-4ba4-8e23-cfa0f06e38ce None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:41.587807 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250cb95a-de19-4ba4-8e23-cfa0f06e38ce None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:41.588677 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250cb95a-de19-4ba4-8e23-cfa0f06e38ce None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:41.588989 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250cb95a-de19-4ba4-8e23-cfa0f06e38ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:41.590492 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250cb95a-de19-4ba4-8e23-cfa0f06e38ce None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:41.618011 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 292/597] 10.222.0.71 () {56 vars in 1137 bytes} [Mon Jul 4 09:06:41 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:41.623955 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250cb95a-de19-4ba4-8e23-cfa0f06e38ce None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:41.636989 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2a1980-3cdc-46a3-874b-6314176f79f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:41.637828 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a1980-3cdc-46a3-874b-6314176f79f6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:41.638108 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a1980-3cdc-46a3-874b-6314176f79f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:41.638382 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a1980-3cdc-46a3-874b-6314176f79f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:41.655916 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2a1980-3cdc-46a3-874b-6314176f79f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:41.657141 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2a1980-3cdc-46a3-874b-6314176f79f6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:41.686487 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-2a2a1980-3cdc-46a3-874b-6314176f79f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:41.686487 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:41.686487 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:41.686487 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:41.686487 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:41.686487 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:41.686487 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:41.686487 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:41.686487 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:41.686487 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:41.686487 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:41.686487 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:41.686487 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:06:41.686487 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:06:41.686487 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:06:41.687443 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:06:41.687443 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:41.687443 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:41.687443 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:41.687443 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:41.687443 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:41.687443 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:41.687443 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:41.687443 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:41.687443 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:41.687443 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:41.687443 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:41.687443 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:41.687443 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:41.687443 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:41.687443 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:41.687443 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:41.688437 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:06:41.688437 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:06:41.688437 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:41.688437 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:41.697416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 306/598] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:06:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:41.698291 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2a1980-3cdc-46a3-874b-6314176f79f6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:41.701186 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:41.739404 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250cb95a-de19-4ba4-8e23-cfa0f06e38ce None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:41.777746 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7b4caa-7a20-4332-8d35-817788d99d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:41.778913 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b4caa-7a20-4332-8d35-817788d99d53 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:41.779404 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b4caa-7a20-4332-8d35-817788d99d53 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:41.779908 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b4caa-7a20-4332-8d35-817788d99d53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:41.791918 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7b4caa-7a20-4332-8d35-817788d99d53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:41.793675 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7b4caa-7a20-4332-8d35-817788d99d53 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:41.802868 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:41.814807 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-df7b4caa-7a20-4332-8d35-817788d99d53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:41.814807 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:41.814807 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:41.814807 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:41.814807 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:41.814807 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:41.814807 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:41.814807 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:41.814807 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:41.814807 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:41.814807 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:41.814807 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:41.814807 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:06:41.814807 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:06:41.814807 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:06:41.815745 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:06:41.815745 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:41.815745 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:41.815745 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:41.815745 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:41.815745 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:41.815745 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:41.815745 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:41.815745 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:41.815745 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:41.815745 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:41.815745 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:41.815745 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:41.815745 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:41.815745 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:41.815745 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:41.815745 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:41.816666 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:06:41.816666 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:06:41.816666 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:41.816666 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:41.819014 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 293/599] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:06:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:06:41.825287 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7b4caa-7a20-4332-8d35-817788d99d53 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:41.864746 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2a1980-3cdc-46a3-874b-6314176f79f6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:41.900646 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65971ec-fc24-44f3-bdb2-dd2330c72d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:41.901358 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65971ec-fc24-44f3-bdb2-dd2330c72d76 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:41.901700 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65971ec-fc24-44f3-bdb2-dd2330c72d76 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:41.901916 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65971ec-fc24-44f3-bdb2-dd2330c72d76 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:41.902638 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65971ec-fc24-44f3-bdb2-dd2330c72d76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:41.902945 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65971ec-fc24-44f3-bdb2-dd2330c72d76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:41.904298 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65971ec-fc24-44f3-bdb2-dd2330c72d76 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:41.921879 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 307/600] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:06:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:41.923045 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65971ec-fc24-44f3-bdb2-dd2330c72d76 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:41.941484 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7b4caa-7a20-4332-8d35-817788d99d53 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:41.980751 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685e386d-e4ae-48e1-8db5-2c88389bef65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:41.981948 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e386d-e4ae-48e1-8db5-2c88389bef65 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:41.982534 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e386d-e4ae-48e1-8db5-2c88389bef65 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:41.983018 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e386d-e4ae-48e1-8db5-2c88389bef65 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:41.983949 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685e386d-e4ae-48e1-8db5-2c88389bef65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:41.984461 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685e386d-e4ae-48e1-8db5-2c88389bef65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:41.986371 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685e386d-e4ae-48e1-8db5-2c88389bef65 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:41.995771 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:42.005005 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 294/601] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:06:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:42.011539 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685e386d-e4ae-48e1-8db5-2c88389bef65 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:42.060100 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65971ec-fc24-44f3-bdb2-dd2330c72d76 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:42.080175 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:42.127194 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75ecd39-264d-4a8e-afad-41b5662ef9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:42.128167 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ecd39-264d-4a8e-afad-41b5662ef9ce None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:42.128454 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ecd39-264d-4a8e-afad-41b5662ef9ce None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:42.128734 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ecd39-264d-4a8e-afad-41b5662ef9ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:42.130383 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685e386d-e4ae-48e1-8db5-2c88389bef65 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:42.141947 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75ecd39-264d-4a8e-afad-41b5662ef9ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:42.144155 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75ecd39-264d-4a8e-afad-41b5662ef9ce None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:42.164700 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-c75ecd39-264d-4a8e-afad-41b5662ef9ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:06:42.164700 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:42.164700 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:42.164700 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:42.164700 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:42.164700 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:42.164700 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:42.164700 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:42.164700 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:42.164700 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:42.164700 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:42.164700 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:42.164700 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:06:42.164700 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:06:42.164700 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:06:42.165851 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:06:42.165851 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:42.165851 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:42.165851 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:42.165851 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:42.165851 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:42.165851 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:42.165851 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:42.165851 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:42.165851 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:42.165851 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:42.165851 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:42.165851 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:42.165851 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:42.165851 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:42.165851 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:42.165851 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:42.166740 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:06:42.166740 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:06:42.166740 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:06:42.166740 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:06:42.166740 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:06:42.166740 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:06:42.166740 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:06:42.166740 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:42.166740 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 308/602] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 4 09:06:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:42.174108 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75ecd39-264d-4a8e-afad-41b5662ef9ce None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:42.178114 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414cb1fa-258b-46a6-8e66-420563d7e736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:42.179205 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414cb1fa-258b-46a6-8e66-420563d7e736 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:42.181313 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414cb1fa-258b-46a6-8e66-420563d7e736 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:42.181808 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414cb1fa-258b-46a6-8e66-420563d7e736 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:42.196700 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414cb1fa-258b-46a6-8e66-420563d7e736 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:42.198577 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414cb1fa-258b-46a6-8e66-420563d7e736 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:42.222021 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-414cb1fa-258b-46a6-8e66-420563d7e736 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:06:42.222021 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:42.222021 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:42.222021 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:42.222021 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:42.222021 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:42.222021 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:42.222021 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:42.222021 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:42.222021 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:42.222021 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:42.222021 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:42.222021 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:06:42.222021 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:06:42.222021 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:06:42.222964 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:06:42.222964 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:42.222964 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:42.222964 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:42.222964 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:42.222964 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:42.222964 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:42.222964 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:42.222964 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:42.222964 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:42.222964 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:42.222964 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:42.222964 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:42.222964 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:42.222964 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:42.222964 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:42.222964 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:42.223858 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:06:42.223858 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:06:42.223858 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:06:42.223858 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:06:42.223858 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:06:42.223858 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:06:42.223858 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:06:42.223858 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:42.234599 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 295/603] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 4 09:06:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:42.240974 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414cb1fa-258b-46a6-8e66-420563d7e736 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:42.395329 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75ecd39-264d-4a8e-afad-41b5662ef9ce None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:42.431667 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:42.445198 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414cb1fa-258b-46a6-8e66-420563d7e736 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:42.450238 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c63aa9b-e677-42d3-8bb2-33f07e9cbc8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:42.450920 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c63aa9b-e677-42d3-8bb2-33f07e9cbc8c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:42.451203 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c63aa9b-e677-42d3-8bb2-33f07e9cbc8c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:42.451477 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c63aa9b-e677-42d3-8bb2-33f07e9cbc8c None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:42.452273 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c63aa9b-e677-42d3-8bb2-33f07e9cbc8c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:42.457397 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c63aa9b-e677-42d3-8bb2-33f07e9cbc8c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:42.458848 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c63aa9b-e677-42d3-8bb2-33f07e9cbc8c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:42.479778 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 309/604] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 4 09:06:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:42.486869 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:42.487011 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c63aa9b-e677-42d3-8bb2-33f07e9cbc8c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:42.518129 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23aaa8f-a563-4074-9aa8-28f871704dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:42.518797 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23aaa8f-a563-4074-9aa8-28f871704dd4 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:42.519077 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23aaa8f-a563-4074-9aa8-28f871704dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:42.519348 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23aaa8f-a563-4074-9aa8-28f871704dd4 None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:42.520141 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23aaa8f-a563-4074-9aa8-28f871704dd4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:42.520450 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23aaa8f-a563-4074-9aa8-28f871704dd4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:42.521928 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23aaa8f-a563-4074-9aa8-28f871704dd4 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:42.543150 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 296/605] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 4 09:06:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:42.549202 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23aaa8f-a563-4074-9aa8-28f871704dd4 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:42.650789 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c63aa9b-e677-42d3-8bb2-33f07e9cbc8c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:42.667836 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23aaa8f-a563-4074-9aa8-28f871704dd4 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:42.686206 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cb63db-7a28-4fe2-9117-02d4ad0a414a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:42.687025 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb63db-7a28-4fe2-9117-02d4ad0a414a None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:42.687310 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb63db-7a28-4fe2-9117-02d4ad0a414a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:42.687588 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb63db-7a28-4fe2-9117-02d4ad0a414a None admin] PATH_INFO: `/v3/projects/4076e316243946e980ce47386b18b28f/users/15818e58661646f381da0973d4c2ff64/roles/5752a9bdfe4b482ab37836eba7bd536b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:42.703481 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cb63db-7a28-4fe2-9117-02d4ad0a414a None admin] RBAC: Authorizing `identity:create_grant(project_id=4076e316243946e980ce47386b18b28f, user_id=15818e58661646f381da0973d4c2ff64, role_id=5752a9bdfe4b482ab37836eba7bd536b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:42.704924 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cb63db-7a28-4fe2-9117-02d4ad0a414a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:42.706291 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c84c7e-3c44-47a5-99c9-d1dedb6d6c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:42.707195 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c84c7e-3c44-47a5-99c9-d1dedb6d6c97 None admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:42.707464 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c84c7e-3c44-47a5-99c9-d1dedb6d6c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:42.707746 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c84c7e-3c44-47a5-99c9-d1dedb6d6c97 None admin] PATH_INFO: `/v3/projects/4076e316243946e980ce47386b18b28f/users/f6e50a3715a14714b3951d36338dde42/roles/5752a9bdfe4b482ab37836eba7bd536b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:42.719716 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 310/606] 10.222.0.71 () {58 vars in 1439 bytes} [Mon Jul 4 09:06:42 2022] PUT /identity/v3/projects/4076e316243946e980ce47386b18b28f/users/15818e58661646f381da0973d4c2ff64/roles/5752a9bdfe4b482ab37836eba7bd536b => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:06:42.721323 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f3a38-ea45-4254-b567-1c8ef1a10dbb None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:42.721616 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f3a38-ea45-4254-b567-1c8ef1a10dbb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:42.721894 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f3a38-ea45-4254-b567-1c8ef1a10dbb None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:42.723314 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 311/607] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:42.724340 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73691bc0-0b8a-4e84-b170-e3873acfa394 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:42.724627 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73691bc0-0b8a-4e84-b170-e3873acfa394 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:42.724917 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73691bc0-0b8a-4e84-b170-e3873acfa394 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:42.726405 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 312/608] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:42.732518 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee77f59-be5c-4412-acb2-d85ed2e519c3 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:42.732858 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee77f59-be5c-4412-acb2-d85ed2e519c3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:42.733303 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee77f59-be5c-4412-acb2-d85ed2e519c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:42.738753 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c84c7e-3c44-47a5-99c9-d1dedb6d6c97 None admin] RBAC: Authorizing `identity:create_grant(project_id=4076e316243946e980ce47386b18b28f, user_id=f6e50a3715a14714b3951d36338dde42, role_id=5752a9bdfe4b482ab37836eba7bd536b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:42.740403 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c84c7e-3c44-47a5-99c9-d1dedb6d6c97 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:42.759375 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-dee77f59-be5c-4412-acb2-d85ed2e519c3 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:42.759789 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 297/609] 10.222.0.71 () {58 vars in 1439 bytes} [Mon Jul 4 09:06:42 2022] PUT /identity/v3/projects/4076e316243946e980ce47386b18b28f/users/f6e50a3715a14714b3951d36338dde42/roles/5752a9bdfe4b482ab37836eba7bd536b => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:06:42.761208 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c7d49-da86-4062-8a33-e8084b5e1cde None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:42.761499 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c7d49-da86-4062-8a33-e8084b5e1cde None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:42.761778 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c7d49-da86-4062-8a33-e8084b5e1cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:42.762003 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dee77f59-be5c-4412-acb2-d85ed2e519c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:42.784001 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-ef4c7d49-da86-4062-8a33-e8084b5e1cde None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:42.785479 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ef4c7d49-da86-4062-8a33-e8084b5e1cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:42.798328 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:42.824163 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:42.878604 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 313/610] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:06:42.897620 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 298/611] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:06:42.920816 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee77f59-be5c-4412-acb2-d85ed2e519c3 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:42.922451 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4c7d49-da86-4062-8a33-e8084b5e1cde None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:43.036280 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee77f59-be5c-4412-acb2-d85ed2e519c3 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:43.064213 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4c7d49-da86-4062-8a33-e8084b5e1cde None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:43.075268 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ccbee0-6845-47c9-8985-2619c0ed2415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:43.076084 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ccbee0-6845-47c9-8985-2619c0ed2415 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:43.076400 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ccbee0-6845-47c9-8985-2619c0ed2415 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:43.076715 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ccbee0-6845-47c9-8985-2619c0ed2415 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:43.089119 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ccbee0-6845-47c9-8985-2619c0ed2415 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:06:43.096921 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-56ccbee0-6845-47c9-8985-2619c0ed2415 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:06:43.096921 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:43.096921 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:43.096921 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:43.096921 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:43.096921 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:43.096921 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:43.096921 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:43.096921 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:43.096921 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:43.096921 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:43.096921 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:43.096921 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:06:43.096921 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:06:43.096921 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:06:43.096921 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:06:43.097922 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:06:43.097922 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:06:43.097922 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:06:43.097922 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:06:43.097922 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:43.097922 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:43.097922 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:43.097922 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:43.097922 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:43.097922 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:43.097922 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:43.097922 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:43.097922 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:43.097922 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:43.097922 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:43.097922 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:43.098787 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:43.098787 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:43.098787 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:43.098787 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:43.098787 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:06:43.098787 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:06:43.098787 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:06:43.098787 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:06:43.098787 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:06:43.098787 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:06:43.098787 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:06:43.098787 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:43.098787 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 314/612] 10.222.0.71 () {56 vars in 1102 bytes} [Mon Jul 4 09:06:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:43.105414 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ccbee0-6845-47c9-8985-2619c0ed2415 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:43.105695 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c82ec5-89af-422b-b968-99c4bfcd4504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:43.106511 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c82ec5-89af-422b-b968-99c4bfcd4504 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:43.106797 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c82ec5-89af-422b-b968-99c4bfcd4504 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:43.107074 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c82ec5-89af-422b-b968-99c4bfcd4504 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:43.121881 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c82ec5-89af-422b-b968-99c4bfcd4504 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:06:43.132495 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:43.133467 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-a4c82ec5-89af-422b-b968-99c4bfcd4504 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:06:43.133467 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:43.133467 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:43.133467 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:43.133467 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:43.133467 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:43.133467 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:43.133467 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:43.133467 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:43.133467 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:43.133467 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:43.133467 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:43.133467 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:06:43.133467 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:06:43.133467 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:06:43.133467 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:06:43.135419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:06:43.135419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:06:43.135419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:06:43.135419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:06:43.135419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:43.135419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:43.135419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:43.135419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:43.135419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:43.135419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:43.135419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:43.135419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:43.135419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:43.135419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:43.135419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:43.135419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:43.136573 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:43.136573 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:43.136573 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:43.136573 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:43.136573 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:06:43.136573 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:06:43.136573 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:06:43.136573 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:06:43.136573 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:06:43.136573 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:06:43.136573 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:06:43.136573 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:43.136573 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 299/613] 10.222.0.71 () {56 vars in 1102 bytes} [Mon Jul 4 09:06:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:43.154265 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c82ec5-89af-422b-b968-99c4bfcd4504 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:43.201610 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:43.234215 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ccbee0-6845-47c9-8985-2619c0ed2415 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:43.274550 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21dce6f-3623-425d-a84c-831b871f2bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:43.275244 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21dce6f-3623-425d-a84c-831b871f2bed None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:43.275527 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21dce6f-3623-425d-a84c-831b871f2bed None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:43.275805 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21dce6f-3623-425d-a84c-831b871f2bed None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:43.276560 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21dce6f-3623-425d-a84c-831b871f2bed None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:43.276868 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21dce6f-3623-425d-a84c-831b871f2bed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:43.278675 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21dce6f-3623-425d-a84c-831b871f2bed None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:43.288403 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 315/614] 10.222.0.71 () {56 vars in 1103 bytes} [Mon Jul 4 09:06:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:43.295771 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21dce6f-3623-425d-a84c-831b871f2bed None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:43.376132 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c82ec5-89af-422b-b968-99c4bfcd4504 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:43.440510 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d3eb64-a14e-4503-8a8c-06ea1fc31e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:43.441432 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d3eb64-a14e-4503-8a8c-06ea1fc31e3b None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:43.441718 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d3eb64-a14e-4503-8a8c-06ea1fc31e3b None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:43.441996 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d3eb64-a14e-4503-8a8c-06ea1fc31e3b None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:43.442762 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d3eb64-a14e-4503-8a8c-06ea1fc31e3b None admin] RBAC: Adding query filter params (name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:43.443069 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d3eb64-a14e-4503-8a8c-06ea1fc31e3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:43.451225 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d3eb64-a14e-4503-8a8c-06ea1fc31e3b None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:43.469834 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:43.471091 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 300/615] 10.222.0.71 () {56 vars in 1103 bytes} [Mon Jul 4 09:06:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:06:43.476686 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d3eb64-a14e-4503-8a8c-06ea1fc31e3b None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:43.493380 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21dce6f-3623-425d-a84c-831b871f2bed None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:43.558216 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3987b8-7652-4079-9447-296f4f6770f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:43.558934 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3987b8-7652-4079-9447-296f4f6770f7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:43.559219 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3987b8-7652-4079-9447-296f4f6770f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:43.559498 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3987b8-7652-4079-9447-296f4f6770f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:43.578952 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3987b8-7652-4079-9447-296f4f6770f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:43.581537 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3987b8-7652-4079-9447-296f4f6770f7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:43.618584 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-8a3987b8-7652-4079-9447-296f4f6770f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:43.618584 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:43.618584 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:43.618584 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:43.618584 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:43.618584 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:43.618584 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:43.618584 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:43.618584 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:43.618584 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:43.618584 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:43.618584 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:43.618584 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:06:43.618584 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:06:43.618584 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:06:43.619555 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:06:43.619555 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:43.619555 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:43.619555 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:43.619555 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:43.619555 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:43.619555 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:43.619555 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:43.619555 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:43.619555 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:43.619555 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:43.619555 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:43.619555 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:43.619555 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:43.619555 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:43.619555 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:43.619555 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:43.620454 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:06:43.620454 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:06:43.620454 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:43.620454 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:43.623865 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 316/616] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:06:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:43.630706 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3987b8-7652-4079-9447-296f4f6770f7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:43.637849 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:43.672578 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d3eb64-a14e-4503-8a8c-06ea1fc31e3b None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:43.740335 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b290616-b368-4006-9df2-43ad54419e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:43.741168 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b290616-b368-4006-9df2-43ad54419e75 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:43.741779 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b290616-b368-4006-9df2-43ad54419e75 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:43.741779 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b290616-b368-4006-9df2-43ad54419e75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:43.766509 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b290616-b368-4006-9df2-43ad54419e75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:43.768052 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b290616-b368-4006-9df2-43ad54419e75 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:43.795295 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-2b290616-b368-4006-9df2-43ad54419e75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:43.795295 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:43.795295 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:43.795295 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:43.795295 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:43.795295 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:43.795295 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:43.795295 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:43.795295 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:43.795295 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:43.795295 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:43.795295 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:43.795295 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:06:43.795295 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:06:43.795295 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:06:43.797509 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:06:43.797509 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:43.797509 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:43.797509 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:43.797509 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:43.797509 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:43.797509 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:43.797509 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:43.797509 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:43.797509 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:43.797509 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:43.797509 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:43.797509 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:43.797509 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:43.797509 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:43.797509 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:43.797509 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:43.798830 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:06:43.798830 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:06:43.798830 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:43.798830 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:43.798830 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 301/617] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:06:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:43.802109 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b290616-b368-4006-9df2-43ad54419e75 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:43.824087 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3987b8-7652-4079-9447-296f4f6770f7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:43.886620 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d2c334-7f5a-447a-9d6e-3991c7509ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:43.887353 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d2c334-7f5a-447a-9d6e-3991c7509ecd None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:43.887642 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d2c334-7f5a-447a-9d6e-3991c7509ecd None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:43.887924 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d2c334-7f5a-447a-9d6e-3991c7509ecd None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:43.889879 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d2c334-7f5a-447a-9d6e-3991c7509ecd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:43.890277 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d2c334-7f5a-447a-9d6e-3991c7509ecd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:43.892900 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d2c334-7f5a-447a-9d6e-3991c7509ecd None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:43.900450 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:43.918764 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 317/618] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:06:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:43.926646 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d2c334-7f5a-447a-9d6e-3991c7509ecd None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:43.988111 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b290616-b368-4006-9df2-43ad54419e75 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:44.042587 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b067d3-6211-413e-a96c-bc8da5d7e94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:44.043261 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b067d3-6211-413e-a96c-bc8da5d7e94f None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:44.043547 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b067d3-6211-413e-a96c-bc8da5d7e94f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:44.043827 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b067d3-6211-413e-a96c-bc8da5d7e94f None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:44.044555 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b067d3-6211-413e-a96c-bc8da5d7e94f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:44.044866 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b067d3-6211-413e-a96c-bc8da5d7e94f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:44.046305 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b067d3-6211-413e-a96c-bc8da5d7e94f None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:44.047802 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:44.064526 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 302/619] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:06:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:44.066871 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2187ce9-94f6-42ee-b2d0-0991cf660ca0 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:44.067615 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2187ce9-94f6-42ee-b2d0-0991cf660ca0 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:44.068123 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2187ce9-94f6-42ee-b2d0-0991cf660ca0 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:44.074939 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 303/620] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:44 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:44.074939 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2187ce9-94f6-42ee-b2d0-0991cf660ca0 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:44.082413 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d2c334-7f5a-447a-9d6e-3991c7509ecd None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:44.138776 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870bdea8-37b3-4c2e-a4f2-aa31d77588aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:44.140566 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870bdea8-37b3-4c2e-a4f2-aa31d77588aa None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:44.141311 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870bdea8-37b3-4c2e-a4f2-aa31d77588aa None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:44.141822 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870bdea8-37b3-4c2e-a4f2-aa31d77588aa None admin] PATH_INFO: `/v3/users/swift` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:44.153519 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870bdea8-37b3-4c2e-a4f2-aa31d77588aa None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:44.156348 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870bdea8-37b3-4c2e-a4f2-aa31d77588aa None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:44.177487 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-870bdea8-37b3-4c2e-a4f2-aa31d77588aa None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 04 09:06:44.177487 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:44.177487 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:44.177487 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:44.177487 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:44.177487 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:44.177487 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:44.177487 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:44.177487 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:44.177487 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:44.177487 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:44.177487 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:44.177487 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:06:44.177487 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:06:44.177487 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:06:44.177487 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:06:44.178902 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:44.178902 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:44.178902 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:06:44.178902 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:06:44.178902 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:06:44.178902 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:06:44.178902 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:44.178902 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:44.178902 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:44.178902 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:44.178902 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:44.178902 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:44.178902 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:44.178902 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:44.178902 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:44.178902 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:44.178902 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:44.178902 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:44.180165 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:44.180165 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:44.180165 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:06:44.180165 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:06:44.180165 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:06:44.180165 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:06:44.180165 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:06:44.180165 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:06:44.180165 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 04 09:06:44.180165 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:44.183067 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 318/621] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 4 09:06:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:44.184949 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d26a7e-0494-4c5d-baf3-c2f9512b6ce3 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:44.185815 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d26a7e-0494-4c5d-baf3-c2f9512b6ce3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:44.186256 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d26a7e-0494-4c5d-baf3-c2f9512b6ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:44.192359 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2187ce9-94f6-42ee-b2d0-0991cf660ca0 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:44.209820 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-89d26a7e-0494-4c5d-baf3-c2f9512b6ce3 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:44.213323 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-89d26a7e-0494-4c5d-baf3-c2f9512b6ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:44.235781 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478f9d38-1c75-4959-82b5-ceaffc9c2070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:44.236610 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478f9d38-1c75-4959-82b5-ceaffc9c2070 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:44.236903 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478f9d38-1c75-4959-82b5-ceaffc9c2070 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:44.237495 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478f9d38-1c75-4959-82b5-ceaffc9c2070 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:44.247178 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478f9d38-1c75-4959-82b5-ceaffc9c2070 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:44.249371 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478f9d38-1c75-4959-82b5-ceaffc9c2070 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:44.267460 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-478f9d38-1c75-4959-82b5-ceaffc9c2070 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 04 09:06:44.267460 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:44.267460 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:44.267460 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:44.267460 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:44.267460 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:44.267460 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:44.267460 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:44.267460 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:44.267460 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:44.267460 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:44.267460 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:44.267460 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:06:44.267460 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:06:44.267460 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:06:44.267460 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:06:44.268682 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:44.268682 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:44.268682 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:06:44.268682 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:06:44.268682 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:06:44.268682 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:06:44.268682 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:44.268682 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:44.268682 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:44.268682 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:44.268682 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:44.268682 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:44.268682 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:44.268682 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:44.268682 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:44.268682 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:44.268682 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:44.268682 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:44.269895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:44.269895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:44.269895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:06:44.269895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:06:44.269895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:06:44.269895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:06:44.269895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:06:44.269895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:06:44.269895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 04 09:06:44.269895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:44.269895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 304/622] 10.222.0.71 () {56 vars in 1093 bytes} [Mon Jul 4 09:06:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:44.269895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478f9d38-1c75-4959-82b5-ceaffc9c2070 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:44.282344 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:44.313379 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 319/623] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:06:44.313379 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d26a7e-0494-4c5d-baf3-c2f9512b6ce3 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:44.356401 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:44.412717 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478f9d38-1c75-4959-82b5-ceaffc9c2070 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:44.451340 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef5f32a-bda5-4208-bbad-c338cb1771ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:44.452228 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef5f32a-bda5-4208-bbad-c338cb1771ec None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:44.452615 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d26a7e-0494-4c5d-baf3-c2f9512b6ce3 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:44.452867 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef5f32a-bda5-4208-bbad-c338cb1771ec None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:44.453343 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef5f32a-bda5-4208-bbad-c338cb1771ec None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:44.454615 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef5f32a-bda5-4208-bbad-c338cb1771ec None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:44.455892 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef5f32a-bda5-4208-bbad-c338cb1771ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:44.457523 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef5f32a-bda5-4208-bbad-c338cb1771ec None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:44.477766 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 305/624] 10.222.0.71 () {56 vars in 1135 bytes} [Mon Jul 4 09:06:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:44.478712 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef5f32a-bda5-4208-bbad-c338cb1771ec None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:44.491336 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d244dce-12f3-447b-a6af-ceb4634c4280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:44.492920 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d244dce-12f3-447b-a6af-ceb4634c4280 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:44.493395 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d244dce-12f3-447b-a6af-ceb4634c4280 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:44.493852 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d244dce-12f3-447b-a6af-ceb4634c4280 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:44.494849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d244dce-12f3-447b-a6af-ceb4634c4280 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:44.495345 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d244dce-12f3-447b-a6af-ceb4634c4280 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:44.496918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d244dce-12f3-447b-a6af-ceb4634c4280 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:44.515636 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 320/625] 10.222.0.71 () {56 vars in 1133 bytes} [Mon Jul 4 09:06:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:44.516726 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d244dce-12f3-447b-a6af-ceb4634c4280 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:44.565943 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:44.584400 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef5f32a-bda5-4208-bbad-c338cb1771ec None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:44.599335 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:44.622229 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653e3909-54bd-4489-b015-2a32af2e52aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:44.622962 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e3909-54bd-4489-b015-2a32af2e52aa None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:44.623376 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e3909-54bd-4489-b015-2a32af2e52aa None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:44.623499 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e3909-54bd-4489-b015-2a32af2e52aa None admin] PATH_INFO: `/v3/roles/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:44.629465 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d244dce-12f3-447b-a6af-ceb4634c4280 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:44.634962 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653e3909-54bd-4489-b015-2a32af2e52aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:06:44.642702 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-653e3909-54bd-4489-b015-2a32af2e52aa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:06:44.642702 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:44.642702 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:44.642702 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:44.642702 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:44.642702 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:44.642702 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:44.642702 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:44.642702 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:44.642702 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:44.642702 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:44.642702 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:44.642702 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:06:44.642702 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:06:44.642702 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:06:44.642702 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:06:44.643889 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:06:44.643889 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:06:44.643889 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:06:44.643889 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:06:44.643889 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:44.643889 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:44.643889 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:44.643889 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:44.643889 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:44.643889 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:44.643889 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:44.643889 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:44.643889 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:44.643889 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:44.643889 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:44.643889 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:44.644891 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:44.644891 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:44.644891 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:44.644891 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:44.644891 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:06:44.644891 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:06:44.644891 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:06:44.644891 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:06:44.644891 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:06:44.644891 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:06:44.644891 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:06:44.644891 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:44.649217 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 306/626] 10.222.0.71 () {56 vars in 1102 bytes} [Mon Jul 4 09:06:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:44.650157 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653e3909-54bd-4489-b015-2a32af2e52aa None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:44.672294 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee58c952-0929-45ba-afca-dad57773fa81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:44.673385 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee58c952-0929-45ba-afca-dad57773fa81 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:44.673840 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee58c952-0929-45ba-afca-dad57773fa81 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:44.674277 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee58c952-0929-45ba-afca-dad57773fa81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:44.685835 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee58c952-0929-45ba-afca-dad57773fa81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:44.687639 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee58c952-0929-45ba-afca-dad57773fa81 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:44.706918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-ee58c952-0929-45ba-afca-dad57773fa81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:44.706918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:44.706918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:44.706918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:44.706918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:44.706918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:44.706918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:44.706918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:44.706918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:44.706918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:44.706918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:44.706918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:44.706918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:06:44.706918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:06:44.706918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:06:44.708061 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:06:44.708061 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:44.708061 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:44.708061 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:44.708061 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:44.708061 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:44.708061 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:44.708061 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:44.708061 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:44.708061 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:44.708061 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:44.708061 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:44.708061 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:44.708061 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:44.708061 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:44.708061 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:44.708061 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:44.709180 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:06:44.709180 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:06:44.709180 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:44.709180 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:44.711657 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 321/627] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:06:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:44.712887 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee58c952-0929-45ba-afca-dad57773fa81 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:44.755376 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653e3909-54bd-4489-b015-2a32af2e52aa None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:44.793228 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703f4007-760b-4f3f-aa99-574e2e0b4b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:44.793858 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f4007-760b-4f3f-aa99-574e2e0b4b5d None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:44.795638 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f4007-760b-4f3f-aa99-574e2e0b4b5d None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:44.795638 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f4007-760b-4f3f-aa99-574e2e0b4b5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:44.821110 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:44.828317 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee58c952-0929-45ba-afca-dad57773fa81 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:44.848366 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:44.874488 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db1db2c-55c8-4590-99b6-cbc4c2dd677e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:44.875617 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db1db2c-55c8-4590-99b6-cbc4c2dd677e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:44.876097 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db1db2c-55c8-4590-99b6-cbc4c2dd677e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:44.876554 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db1db2c-55c8-4590-99b6-cbc4c2dd677e None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:44.877533 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db1db2c-55c8-4590-99b6-cbc4c2dd677e None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:44.878038 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db1db2c-55c8-4590-99b6-cbc4c2dd677e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:44.879794 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db1db2c-55c8-4590-99b6-cbc4c2dd677e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:44.882602 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703f4007-760b-4f3f-aa99-574e2e0b4b5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:44.884793 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703f4007-760b-4f3f-aa99-574e2e0b4b5d None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:44.891145 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 322/628] 10.222.0.71 () {56 vars in 1103 bytes} [Mon Jul 4 09:06:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:44.892195 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db1db2c-55c8-4590-99b6-cbc4c2dd677e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:45.009217 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 307/629] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:06:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:06:45.010451 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703f4007-760b-4f3f-aa99-574e2e0b4b5d None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:45.022835 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db1db2c-55c8-4590-99b6-cbc4c2dd677e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:45.071901 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f41928-b478-42e4-baf0-b76a31d094f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:45.073139 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f41928-b478-42e4-baf0-b76a31d094f5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:45.073514 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f41928-b478-42e4-baf0-b76a31d094f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:45.073857 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f41928-b478-42e4-baf0-b76a31d094f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:45.074947 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f41928-b478-42e4-baf0-b76a31d094f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:45.075323 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f41928-b478-42e4-baf0-b76a31d094f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:45.077451 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f41928-b478-42e4-baf0-b76a31d094f5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:45.095211 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:45.095573 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 323/630] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:06:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:45.097018 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f41928-b478-42e4-baf0-b76a31d094f5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:45.122610 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:45.133481 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703f4007-760b-4f3f-aa99-574e2e0b4b5d None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:45.179531 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a70aef-154f-4ef3-aa2d-c8f5b4f08b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:45.181565 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a70aef-154f-4ef3-aa2d-c8f5b4f08b31 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:45.181849 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a70aef-154f-4ef3-aa2d-c8f5b4f08b31 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:45.182132 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a70aef-154f-4ef3-aa2d-c8f5b4f08b31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:45.192925 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a70aef-154f-4ef3-aa2d-c8f5b4f08b31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:45.194418 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a70aef-154f-4ef3-aa2d-c8f5b4f08b31 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:45.212732 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-45a70aef-154f-4ef3-aa2d-c8f5b4f08b31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:45.212732 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:45.212732 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:45.212732 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:45.212732 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:45.212732 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:45.212732 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:45.212732 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:45.212732 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:45.212732 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:45.212732 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:45.212732 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:45.212732 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:06:45.212732 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:06:45.212732 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:06:45.213680 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:06:45.213680 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:45.213680 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:45.213680 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:45.213680 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:45.213680 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:45.213680 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:45.213680 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:45.213680 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:45.213680 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:45.213680 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:45.213680 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:45.213680 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:45.213680 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:45.213680 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:45.213680 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:45.213680 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:45.214931 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:06:45.214931 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:06:45.214931 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:45.214931 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:45.215501 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f41928-b478-42e4-baf0-b76a31d094f5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:45.215587 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 308/631] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:06:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:45.215587 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a70aef-154f-4ef3-aa2d-c8f5b4f08b31 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:45.251723 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e3b6eb-486c-4078-b064-20761f31d6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:45.252514 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e3b6eb-486c-4078-b064-20761f31d6a5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:45.252801 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e3b6eb-486c-4078-b064-20761f31d6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:45.253104 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e3b6eb-486c-4078-b064-20761f31d6a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:45.263325 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e3b6eb-486c-4078-b064-20761f31d6a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:45.264749 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e3b6eb-486c-4078-b064-20761f31d6a5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:45.281776 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-43e3b6eb-486c-4078-b064-20761f31d6a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:45.281776 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:45.281776 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:45.281776 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:45.281776 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:45.281776 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:45.281776 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:45.281776 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:45.281776 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:45.281776 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:45.281776 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:45.281776 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:45.281776 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:06:45.281776 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:06:45.281776 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:06:45.282774 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:06:45.282774 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:45.282774 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:45.282774 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:45.282774 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:45.282774 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:45.282774 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:45.282774 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:45.282774 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:45.282774 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:45.282774 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:45.282774 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:45.282774 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:45.282774 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:45.282774 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:45.282774 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:45.282774 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:45.283683 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:06:45.283683 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:06:45.283683 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:45.283683 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:45.283683 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 324/632] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:06:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:45.284222 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ee15d-e3c2-4d25-89c1-25437fc4a663 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:45.284525 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ee15d-e3c2-4d25-89c1-25437fc4a663 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:45.284781 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ee15d-e3c2-4d25-89c1-25437fc4a663 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:45.286060 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 325/633] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:45.287123 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a2a32d-f9ac-4087-91e3-7f12bfa99e21 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:45.287644 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a2a32d-f9ac-4087-91e3-7f12bfa99e21 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:45.287644 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a2a32d-f9ac-4087-91e3-7f12bfa99e21 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:45.288849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 326/634] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:45.290273 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a2a32d-f9ac-4087-91e3-7f12bfa99e21 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:45.319566 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a70aef-154f-4ef3-aa2d-c8f5b4f08b31 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:45.344417 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:45.355862 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54039038-1d6b-4d6f-85d7-865f4830d582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:45.356651 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54039038-1d6b-4d6f-85d7-865f4830d582 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:45.356932 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54039038-1d6b-4d6f-85d7-865f4830d582 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:45.357239 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54039038-1d6b-4d6f-85d7-865f4830d582 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:45.362417 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:45.363971 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54039038-1d6b-4d6f-85d7-865f4830d582 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:45.365488 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54039038-1d6b-4d6f-85d7-865f4830d582 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:45.379613 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-54039038-1d6b-4d6f-85d7-865f4830d582 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:06:45.379613 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:45.379613 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:45.379613 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:45.379613 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:45.379613 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:45.379613 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:45.379613 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:45.379613 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:45.379613 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:45.379613 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:45.379613 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:45.379613 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:06:45.379613 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:06:45.379613 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:06:45.380624 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:06:45.380624 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:45.380624 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:45.380624 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:45.380624 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:45.380624 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:45.380624 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:45.380624 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:45.380624 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:45.380624 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:45.380624 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:45.380624 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:45.380624 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:45.380624 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:45.380624 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:45.380624 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:45.380624 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:45.384593 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:06:45.384593 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:06:45.384593 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:06:45.384593 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:06:45.384593 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:06:45.384593 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:06:45.384593 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:06:45.384593 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:45.384593 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 309/635] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 4 09:06:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:45.384593 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54039038-1d6b-4d6f-85d7-865f4830d582 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:45.402941 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a2a32d-f9ac-4087-91e3-7f12bfa99e21 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:45.438393 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f54421-465a-4d1c-a1eb-2397e75f87f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:45.439051 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f54421-465a-4d1c-a1eb-2397e75f87f4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:45.439361 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f54421-465a-4d1c-a1eb-2397e75f87f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:45.439639 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f54421-465a-4d1c-a1eb-2397e75f87f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:45.440353 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f54421-465a-4d1c-a1eb-2397e75f87f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:45.440656 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f54421-465a-4d1c-a1eb-2397e75f87f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:45.442052 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f54421-465a-4d1c-a1eb-2397e75f87f4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:45.454281 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 327/636] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:06:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:45.455264 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28dd92e-0716-4a57-bb0e-620d871a4066 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:45.455552 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28dd92e-0716-4a57-bb0e-620d871a4066 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:45.455829 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28dd92e-0716-4a57-bb0e-620d871a4066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:45.473902 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c28dd92e-0716-4a57-bb0e-620d871a4066 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:45.475115 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c28dd92e-0716-4a57-bb0e-620d871a4066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:45.484412 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54039038-1d6b-4d6f-85d7-865f4830d582 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:45.517907 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149faafd-a3bc-4e28-9e7a-1153f81a0d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:45.518569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149faafd-a3bc-4e28-9e7a-1153f81a0d8e None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:45.518855 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149faafd-a3bc-4e28-9e7a-1153f81a0d8e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:45.519136 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149faafd-a3bc-4e28-9e7a-1153f81a0d8e None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:45.519851 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149faafd-a3bc-4e28-9e7a-1153f81a0d8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:45.520162 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149faafd-a3bc-4e28-9e7a-1153f81a0d8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:45.521584 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149faafd-a3bc-4e28-9e7a-1153f81a0d8e None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:45.533741 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 310/637] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:06:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:45.538356 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169c1fd-b6f4-4b1b-829a-e42fa2f0e5ae None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:45.538356 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169c1fd-b6f4-4b1b-829a-e42fa2f0e5ae None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:45.538356 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169c1fd-b6f4-4b1b-829a-e42fa2f0e5ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:45.542093 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 328/638] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:06:45.542930 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28dd92e-0716-4a57-bb0e-620d871a4066 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:45.552999 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-7169c1fd-b6f4-4b1b-829a-e42fa2f0e5ae None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:45.554319 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7169c1fd-b6f4-4b1b-829a-e42fa2f0e5ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:45.594239 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:45.614706 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:45.622801 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 311/639] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:06:45.623624 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7169c1fd-b6f4-4b1b-829a-e42fa2f0e5ae None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:45.656497 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28dd92e-0716-4a57-bb0e-620d871a4066 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:45.690050 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9329bc1-92d8-4c11-bb2c-2370a0159a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:45.690714 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9329bc1-92d8-4c11-bb2c-2370a0159a57 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:45.691001 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9329bc1-92d8-4c11-bb2c-2370a0159a57 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:45.691281 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9329bc1-92d8-4c11-bb2c-2370a0159a57 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:45.692068 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9329bc1-92d8-4c11-bb2c-2370a0159a57 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:45.692380 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9329bc1-92d8-4c11-bb2c-2370a0159a57 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:45.693854 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9329bc1-92d8-4c11-bb2c-2370a0159a57 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:45.705184 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 329/640] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 4 09:06:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:45.705976 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9329bc1-92d8-4c11-bb2c-2370a0159a57 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:45.736172 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7169c1fd-b6f4-4b1b-829a-e42fa2f0e5ae None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:45.769243 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b847c79-45a1-4f4a-8cde-3bbc86255bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:45.770035 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b847c79-45a1-4f4a-8cde-3bbc86255bb9 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:45.770313 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b847c79-45a1-4f4a-8cde-3bbc86255bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:45.770586 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b847c79-45a1-4f4a-8cde-3bbc86255bb9 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:45.778658 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b847c79-45a1-4f4a-8cde-3bbc86255bb9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:45.780122 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b847c79-45a1-4f4a-8cde-3bbc86255bb9 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:45.795648 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-5b847c79-45a1-4f4a-8cde-3bbc86255bb9 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 04 09:06:45.795648 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:45.795648 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:45.795648 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:45.795648 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:45.795648 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:45.795648 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:45.795648 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:45.795648 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:45.795648 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:45.795648 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:45.795648 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:45.795648 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:06:45.795648 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:06:45.795648 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:06:45.795648 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:06:45.796843 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:45.796843 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:45.796843 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:06:45.796843 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:06:45.796843 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:06:45.796843 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:06:45.796843 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:45.796843 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:45.796843 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:45.796843 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:45.796843 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:45.796843 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:45.796843 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:45.796843 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:45.796843 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:45.796843 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:45.796843 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:45.796843 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:45.797869 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:45.797869 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:45.797869 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:06:45.797869 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:06:45.797869 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:06:45.797869 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:06:45.797869 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:06:45.797869 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:06:45.797869 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 04 09:06:45.797869 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:45.797869 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 312/641] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:06:45 2022] GET /identity/v3/users/glance => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:45.797869 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b847c79-45a1-4f4a-8cde-3bbc86255bb9 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:45.810778 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9329bc1-92d8-4c11-bb2c-2370a0159a57 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:45.829878 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:45.830899 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:45.847634 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dae51c-c9f0-44c5-9faf-6f78f47a0bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:45.848432 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dae51c-c9f0-44c5-9faf-6f78f47a0bc0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:45.848718 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dae51c-c9f0-44c5-9faf-6f78f47a0bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:45.848997 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dae51c-c9f0-44c5-9faf-6f78f47a0bc0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:45.856195 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dae51c-c9f0-44c5-9faf-6f78f47a0bc0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:45.857664 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dae51c-c9f0-44c5-9faf-6f78f47a0bc0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:45.871918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-58dae51c-c9f0-44c5-9faf-6f78f47a0bc0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:06:45.871918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:45.871918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:45.871918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:45.871918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:45.871918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:45.871918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:45.871918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:45.871918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:45.871918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:45.871918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:45.871918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:45.871918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:06:45.871918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:06:45.871918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:06:45.872867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:06:45.872867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:45.872867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:45.872867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:45.872867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:45.872867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:45.872867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:45.872867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:45.872867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:45.872867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:45.872867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:45.872867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:45.872867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:45.872867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:45.872867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:45.872867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:45.872867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:45.873889 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:06:45.873889 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:06:45.873889 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:06:45.873889 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:06:45.873889 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:06:45.873889 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:06:45.873889 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:06:45.873889 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:45.873889 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 330/642] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 4 09:06:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:45.874947 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dae51c-c9f0-44c5-9faf-6f78f47a0bc0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:45.900704 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b847c79-45a1-4f4a-8cde-3bbc86255bb9 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:45.936886 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6624b0-1ae9-493d-ab98-db71ade93c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:45.937643 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6624b0-1ae9-493d-ab98-db71ade93c89 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:45.937940 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6624b0-1ae9-493d-ab98-db71ade93c89 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:45.938217 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6624b0-1ae9-493d-ab98-db71ade93c89 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:45.949091 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6624b0-1ae9-493d-ab98-db71ade93c89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:06:45.956757 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-df6624b0-1ae9-493d-ab98-db71ade93c89 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:06:45.956757 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:45.956757 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:45.956757 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:45.956757 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:45.956757 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:45.956757 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:45.956757 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:45.956757 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:45.956757 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:45.956757 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:45.956757 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:45.956757 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:06:45.956757 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:06:45.956757 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:06:45.956757 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:06:45.957925 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:06:45.957925 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:06:45.957925 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:06:45.957925 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:06:45.957925 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:45.957925 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:45.957925 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:45.957925 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:45.957925 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:45.957925 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:45.957925 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:45.957925 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:45.957925 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:45.957925 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:45.957925 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:45.957925 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:45.958864 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:45.958864 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:45.958864 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:45.958864 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:45.958864 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:06:45.958864 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:06:45.958864 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:06:45.958864 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:06:45.958864 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:06:45.958864 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:06:45.958864 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:06:45.958864 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:45.958864 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 313/643] 10.222.0.71 () {56 vars in 1102 bytes} [Mon Jul 4 09:06:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:45.958864 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6624b0-1ae9-493d-ab98-db71ade93c89 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:45.982610 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dae51c-c9f0-44c5-9faf-6f78f47a0bc0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:46.017387 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157314c4-06b1-4e34-ba76-bcbe37301433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:46.018180 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157314c4-06b1-4e34-ba76-bcbe37301433 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:46.018467 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157314c4-06b1-4e34-ba76-bcbe37301433 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:46.018754 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157314c4-06b1-4e34-ba76-bcbe37301433 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:46.029440 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157314c4-06b1-4e34-ba76-bcbe37301433 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:06:46.037155 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-157314c4-06b1-4e34-ba76-bcbe37301433 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:06:46.037155 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:46.037155 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:46.037155 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:46.037155 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:46.037155 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:46.037155 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:46.037155 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:46.037155 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:46.037155 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:46.037155 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:46.037155 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:46.037155 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:06:46.037155 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:06:46.037155 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:06:46.037155 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:06:46.038126 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:06:46.038126 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:06:46.038126 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:06:46.038126 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:06:46.038126 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:46.038126 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:46.038126 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:46.038126 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:46.038126 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:46.038126 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:46.038126 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:46.038126 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:46.038126 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:46.038126 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:46.038126 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:46.038126 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:46.038963 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:46.038963 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:46.038963 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:46.038963 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:46.038963 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:06:46.038963 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:06:46.038963 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:06:46.038963 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:06:46.038963 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:06:46.038963 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:06:46.038963 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:06:46.038963 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:46.038963 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 331/644] 10.222.0.71 () {56 vars in 1102 bytes} [Mon Jul 4 09:06:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:46.039673 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157314c4-06b1-4e34-ba76-bcbe37301433 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:46.053311 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:46.060286 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6624b0-1ae9-493d-ab98-db71ade93c89 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:46.080836 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:46.094692 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376d4999-4904-4824-b9b9-3d60d6a9ba93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:46.095655 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376d4999-4904-4824-b9b9-3d60d6a9ba93 None admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:46.096050 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376d4999-4904-4824-b9b9-3d60d6a9ba93 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:46.096214 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376d4999-4904-4824-b9b9-3d60d6a9ba93 None admin] PATH_INFO: `/v3/projects/4076e316243946e980ce47386b18b28f/users/88b41b55769a4c449716c3590ed58225/roles/5752a9bdfe4b482ab37836eba7bd536b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:46.111121 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376d4999-4904-4824-b9b9-3d60d6a9ba93 None admin] RBAC: Authorizing `identity:create_grant(project_id=4076e316243946e980ce47386b18b28f, user_id=88b41b55769a4c449716c3590ed58225, role_id=5752a9bdfe4b482ab37836eba7bd536b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:46.112572 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376d4999-4904-4824-b9b9-3d60d6a9ba93 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:46.125195 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 314/645] 10.222.0.71 () {58 vars in 1439 bytes} [Mon Jul 4 09:06:45 2022] PUT /identity/v3/projects/4076e316243946e980ce47386b18b28f/users/88b41b55769a4c449716c3590ed58225/roles/5752a9bdfe4b482ab37836eba7bd536b => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:06:46.125995 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376d4999-4904-4824-b9b9-3d60d6a9ba93 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:46.156360 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157314c4-06b1-4e34-ba76-bcbe37301433 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:46.198012 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14691c-5b17-4ade-8d9e-09c6430f2f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:46.198732 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14691c-5b17-4ade-8d9e-09c6430f2f67 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:46.199022 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14691c-5b17-4ade-8d9e-09c6430f2f67 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:46.199301 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14691c-5b17-4ade-8d9e-09c6430f2f67 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:46.200159 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e14691c-5b17-4ade-8d9e-09c6430f2f67 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:46.200473 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e14691c-5b17-4ade-8d9e-09c6430f2f67 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:46.202028 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e14691c-5b17-4ade-8d9e-09c6430f2f67 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:46.223113 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 332/646] 10.222.0.71 () {56 vars in 1137 bytes} [Mon Jul 4 09:06:46 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:46.224067 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14691c-5b17-4ade-8d9e-09c6430f2f67 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:46.246605 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376d4999-4904-4824-b9b9-3d60d6a9ba93 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:46.288145 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc3bc7f-12f4-4afd-8ccb-7eae18e981e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:46.288825 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3bc7f-12f4-4afd-8ccb-7eae18e981e9 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:46.289732 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3bc7f-12f4-4afd-8ccb-7eae18e981e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:46.290480 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3bc7f-12f4-4afd-8ccb-7eae18e981e9 None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:46.290801 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc3bc7f-12f4-4afd-8ccb-7eae18e981e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:46.291116 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc3bc7f-12f4-4afd-8ccb-7eae18e981e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:46.292796 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc3bc7f-12f4-4afd-8ccb-7eae18e981e9 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:46.305949 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 315/647] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 4 09:06:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:46.306791 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc3bc7f-12f4-4afd-8ccb-7eae18e981e9 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:46.311538 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:46.346043 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:46.350901 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14691c-5b17-4ade-8d9e-09c6430f2f67 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:46.399076 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18888a9-6b2f-4264-b040-38924ea21e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:46.400370 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18888a9-6b2f-4264-b040-38924ea21e99 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:46.400671 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18888a9-6b2f-4264-b040-38924ea21e99 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:46.401130 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18888a9-6b2f-4264-b040-38924ea21e99 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:46.401719 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18888a9-6b2f-4264-b040-38924ea21e99 None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:46.402024 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18888a9-6b2f-4264-b040-38924ea21e99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:46.403379 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18888a9-6b2f-4264-b040-38924ea21e99 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:46.416395 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 333/648] 10.222.0.71 () {56 vars in 1103 bytes} [Mon Jul 4 09:06:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:46.416395 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18888a9-6b2f-4264-b040-38924ea21e99 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:46.420969 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc3bc7f-12f4-4afd-8ccb-7eae18e981e9 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:46.456376 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d400e2-28b6-4c09-a8ff-cceb1796221f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:46.457065 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d400e2-28b6-4c09-a8ff-cceb1796221f None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:46.457349 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d400e2-28b6-4c09-a8ff-cceb1796221f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:46.457625 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d400e2-28b6-4c09-a8ff-cceb1796221f None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:46.458365 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d400e2-28b6-4c09-a8ff-cceb1796221f None admin] RBAC: Adding query filter params (name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:46.458669 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d400e2-28b6-4c09-a8ff-cceb1796221f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:46.460037 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d400e2-28b6-4c09-a8ff-cceb1796221f None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:46.473819 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 316/649] 10.222.0.71 () {56 vars in 1103 bytes} [Mon Jul 4 09:06:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:46.473819 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d400e2-28b6-4c09-a8ff-cceb1796221f None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:46.526714 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18888a9-6b2f-4264-b040-38924ea21e99 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:46.544447 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:46.563495 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bcd16f-492c-429a-a128-9f1aa541a9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:46.564158 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bcd16f-492c-429a-a128-9f1aa541a9c7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:46.564445 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bcd16f-492c-429a-a128-9f1aa541a9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:46.564732 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bcd16f-492c-429a-a128-9f1aa541a9c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:46.584476 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d400e2-28b6-4c09-a8ff-cceb1796221f None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:46.599995 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:46.621622 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72fe49f-2446-46ce-a22b-29eb29cea45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:46.622411 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72fe49f-2446-46ce-a22b-29eb29cea45c None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:46.622764 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72fe49f-2446-46ce-a22b-29eb29cea45c None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:46.623052 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72fe49f-2446-46ce-a22b-29eb29cea45c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:46.623886 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72fe49f-2446-46ce-a22b-29eb29cea45c None admin] RBAC: Adding query filter params (role.id=5752a9bdfe4b482ab37836eba7bd536b, scope.project.id=4076e316243946e980ce47386b18b28f, user.id=15039da209ec445d93d49e2bd6d854eb) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:46.624201 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72fe49f-2446-46ce-a22b-29eb29cea45c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:46.625683 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72fe49f-2446-46ce-a22b-29eb29cea45c None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:46.632067 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 317/650] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 4 09:06:46 2022] GET /identity/v3/role_assignments?user.id=15039da209ec445d93d49e2bd6d854eb&scope.project.id=4076e316243946e980ce47386b18b28f&role.id=5752a9bdfe4b482ab37836eba7bd536b => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:46.633486 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72fe49f-2446-46ce-a22b-29eb29cea45c None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:46.656861 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bcd16f-492c-429a-a128-9f1aa541a9c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:46.658377 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bcd16f-492c-429a-a128-9f1aa541a9c7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:46.774482 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 334/651] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:06:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:06:46.774482 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bcd16f-492c-429a-a128-9f1aa541a9c7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:46.792980 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72fe49f-2446-46ce-a22b-29eb29cea45c None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:46.839503 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:46.845840 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ad25c-ad5c-436d-ab53-5d96ddf38841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:46.846490 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:46.847428 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ad25c-ad5c-436d-ab53-5d96ddf38841 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:46.847775 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ad25c-ad5c-436d-ab53-5d96ddf38841 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:46.847998 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ad25c-ad5c-436d-ab53-5d96ddf38841 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:46.865069 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79ad25c-ad5c-436d-ab53-5d96ddf38841 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:46.865069 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79ad25c-ad5c-436d-ab53-5d96ddf38841 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:46.884585 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-c79ad25c-ad5c-436d-ab53-5d96ddf38841 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:46.884585 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:46.884585 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:46.884585 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:46.884585 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:46.884585 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:46.884585 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:46.884585 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:46.884585 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:46.884585 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:46.884585 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:46.884585 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:46.884585 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:06:46.884585 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:06:46.884585 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:06:46.885791 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:06:46.885791 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:46.885791 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:46.885791 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:46.885791 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:46.885791 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:46.885791 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:46.885791 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:46.885791 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:46.885791 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:46.885791 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:46.885791 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:46.885791 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:46.885791 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:46.885791 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:46.885791 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:46.885791 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:46.887661 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:06:46.887661 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:06:46.887661 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:46.887661 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:46.887661 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 318/652] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:06:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:46.887661 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ad25c-ad5c-436d-ab53-5d96ddf38841 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:46.909762 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bcd16f-492c-429a-a128-9f1aa541a9c7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:46.952506 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924bc0ca-8fc4-42dd-be6e-93e5912c7a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:46.953768 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924bc0ca-8fc4-42dd-be6e-93e5912c7a26 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:46.954298 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924bc0ca-8fc4-42dd-be6e-93e5912c7a26 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:46.954782 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924bc0ca-8fc4-42dd-be6e-93e5912c7a26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:46.966910 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924bc0ca-8fc4-42dd-be6e-93e5912c7a26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:46.968639 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924bc0ca-8fc4-42dd-be6e-93e5912c7a26 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:46.987099 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-924bc0ca-8fc4-42dd-be6e-93e5912c7a26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:46.987099 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:46.987099 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:46.987099 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:46.987099 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:46.987099 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:46.987099 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:46.987099 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:46.987099 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:46.987099 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:46.987099 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:46.987099 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:46.987099 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:06:46.987099 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:06:46.987099 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:06:46.988054 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:06:46.988054 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:46.988054 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:46.988054 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:46.988054 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:46.988054 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:46.988054 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:46.988054 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:46.988054 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:46.988054 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:46.988054 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:46.988054 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:46.988054 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:46.988054 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:46.988054 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:46.988054 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:46.988054 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:46.989441 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:06:46.989441 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:06:46.989441 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:46.989441 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:46.991349 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 335/653] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:06:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:46.992517 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924bc0ca-8fc4-42dd-be6e-93e5912c7a26 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:47.010916 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ad25c-ad5c-436d-ab53-5d96ddf38841 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:47.056589 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e0f38-06cd-4960-9811-dbc843d11c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:47.057633 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e0f38-06cd-4960-9811-dbc843d11c90 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:47.057964 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e0f38-06cd-4960-9811-dbc843d11c90 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:47.058494 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e0f38-06cd-4960-9811-dbc843d11c90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:47.074942 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1e0f38-06cd-4960-9811-dbc843d11c90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:47.077340 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1e0f38-06cd-4960-9811-dbc843d11c90 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:47.100299 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-0d1e0f38-06cd-4960-9811-dbc843d11c90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:47.100299 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:47.100299 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:47.100299 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:47.100299 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:47.100299 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:47.100299 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:47.100299 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:47.100299 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:47.100299 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:47.100299 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:47.100299 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:47.100299 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:06:47.100299 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:06:47.100299 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:06:47.101597 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:06:47.101597 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:47.101597 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:47.101597 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:47.101597 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:47.101597 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:47.101597 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:47.101597 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:47.101597 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:47.101597 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:47.101597 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:47.101597 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:47.101597 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:47.101597 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:47.101597 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:47.101597 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:47.101597 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:47.102882 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:06:47.102882 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:06:47.102882 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:47.102882 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:47.102882 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 319/654] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:06:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:47.103307 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e0f38-06cd-4960-9811-dbc843d11c90 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:47.112431 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924bc0ca-8fc4-42dd-be6e-93e5912c7a26 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:47.129454 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:47.157382 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a7f503-5841-4a9f-81f9-4201301d307a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:47.158224 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7f503-5841-4a9f-81f9-4201301d307a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:47.158603 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:47.158723 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7f503-5841-4a9f-81f9-4201301d307a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:47.159090 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7f503-5841-4a9f-81f9-4201301d307a None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:47.159896 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a7f503-5841-4a9f-81f9-4201301d307a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:47.160510 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a7f503-5841-4a9f-81f9-4201301d307a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:47.162159 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a7f503-5841-4a9f-81f9-4201301d307a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:47.185994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 336/655] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:06:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:47.185994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a7f503-5841-4a9f-81f9-4201301d307a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:47.240620 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e0f38-06cd-4960-9811-dbc843d11c90 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:47.279480 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db989d1-03f2-404b-a877-94abf711fa40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:47.280458 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db989d1-03f2-404b-a877-94abf711fa40 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:47.280896 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db989d1-03f2-404b-a877-94abf711fa40 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:47.281426 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db989d1-03f2-404b-a877-94abf711fa40 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:47.282399 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db989d1-03f2-404b-a877-94abf711fa40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:47.282869 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db989d1-03f2-404b-a877-94abf711fa40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:47.284424 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db989d1-03f2-404b-a877-94abf711fa40 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:47.295264 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a7f503-5841-4a9f-81f9-4201301d307a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:47.298156 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 320/656] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:06:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:47.299122 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db989d1-03f2-404b-a877-94abf711fa40 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:47.332412 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a4b32d-9089-462d-a008-5524b32afe21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:47.333348 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a4b32d-9089-462d-a008-5524b32afe21 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:47.333582 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a4b32d-9089-462d-a008-5524b32afe21 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:47.333967 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a4b32d-9089-462d-a008-5524b32afe21 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:47.334798 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a4b32d-9089-462d-a008-5524b32afe21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:47.335212 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a4b32d-9089-462d-a008-5524b32afe21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:47.336721 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a4b32d-9089-462d-a008-5524b32afe21 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:47.353901 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 337/657] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:06:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:47.354927 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a4b32d-9089-462d-a008-5524b32afe21 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:47.395462 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:47.405315 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:47.420762 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db989d1-03f2-404b-a877-94abf711fa40 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:47.463162 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e3bb8a-4b99-4f65-8cf7-0721d8bbdd84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:47.464064 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3bb8a-4b99-4f65-8cf7-0721d8bbdd84 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:47.464360 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3bb8a-4b99-4f65-8cf7-0721d8bbdd84 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:47.464643 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3bb8a-4b99-4f65-8cf7-0721d8bbdd84 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:47.472767 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e3bb8a-4b99-4f65-8cf7-0721d8bbdd84 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:47.474236 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a4b32d-9089-462d-a008-5524b32afe21 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:47.474361 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e3bb8a-4b99-4f65-8cf7-0721d8bbdd84 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:47.490425 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-69e3bb8a-4b99-4f65-8cf7-0721d8bbdd84 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 04 09:06:47.490425 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:47.490425 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:47.490425 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:47.490425 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:47.490425 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:47.490425 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:47.490425 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:47.490425 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:47.490425 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:47.490425 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:47.490425 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:47.490425 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:06:47.490425 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:06:47.490425 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:06:47.490425 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:06:47.491580 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:47.491580 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:47.491580 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:06:47.491580 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:06:47.491580 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:06:47.491580 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:06:47.491580 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:47.491580 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:47.491580 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:47.491580 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:47.491580 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:47.491580 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:47.491580 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:47.491580 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:47.491580 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:47.491580 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:47.491580 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:47.491580 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:47.493317 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:47.493317 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:47.493317 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:06:47.493317 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:06:47.493317 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:06:47.493317 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:06:47.493317 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:06:47.493317 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:06:47.493317 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 04 09:06:47.493317 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:47.493317 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 321/658] 10.222.0.71 () {56 vars in 1102 bytes} [Mon Jul 4 09:06:47 2022] GET /identity/v3/users/neutron => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:47.493317 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e3bb8a-4b99-4f65-8cf7-0721d8bbdd84 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:47.522182 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104abdaa-565c-40a0-b187-566eb9cc57dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:47.524082 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104abdaa-565c-40a0-b187-566eb9cc57dd None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:47.524520 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104abdaa-565c-40a0-b187-566eb9cc57dd None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:47.524971 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104abdaa-565c-40a0-b187-566eb9cc57dd None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:47.534880 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104abdaa-565c-40a0-b187-566eb9cc57dd None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:47.537437 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104abdaa-565c-40a0-b187-566eb9cc57dd None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:47.555653 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-104abdaa-565c-40a0-b187-566eb9cc57dd None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 04 09:06:47.555653 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:47.555653 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:47.555653 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:47.555653 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:47.555653 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:47.555653 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:47.555653 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:47.555653 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:47.555653 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:47.555653 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:47.555653 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:47.555653 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:06:47.555653 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:06:47.555653 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:06:47.555653 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:06:47.556832 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:47.556832 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:47.556832 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:06:47.556832 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:06:47.556832 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:06:47.556832 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:06:47.556832 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:47.556832 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:47.556832 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:47.556832 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:47.556832 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:47.556832 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:47.556832 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:47.556832 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:47.556832 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:47.556832 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:47.556832 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:47.556832 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:47.558030 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:47.558030 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:47.558030 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:06:47.558030 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:06:47.558030 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:06:47.558030 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:06:47.558030 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:06:47.558030 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:06:47.558030 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 04 09:06:47.558030 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:47.564531 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 338/659] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:06:47 2022] GET /identity/v3/users/cinder => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:47.564531 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104abdaa-565c-40a0-b187-566eb9cc57dd None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:47.614818 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e3bb8a-4b99-4f65-8cf7-0721d8bbdd84 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:47.652565 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e64ee-4422-4986-8b98-b42e97747afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:47.653515 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e64ee-4422-4986-8b98-b42e97747afd None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:47.653902 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e64ee-4422-4986-8b98-b42e97747afd None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:47.654304 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e64ee-4422-4986-8b98-b42e97747afd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:47.661698 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1e64ee-4422-4986-8b98-b42e97747afd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:47.663416 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1e64ee-4422-4986-8b98-b42e97747afd None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:47.676148 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:47.679398 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-ed1e64ee-4422-4986-8b98-b42e97747afd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:06:47.679398 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:47.679398 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:47.679398 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:47.679398 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:47.679398 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:47.679398 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:47.679398 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:47.679398 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:47.679398 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:47.679398 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:47.679398 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:47.679398 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:06:47.679398 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:06:47.679398 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:06:47.680773 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:06:47.680773 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:47.680773 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:47.680773 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:47.680773 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:47.680773 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:47.680773 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:47.680773 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:47.680773 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:47.680773 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:47.680773 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:47.680773 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:47.680773 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:47.680773 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:47.680773 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:47.680773 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:47.680773 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:47.681790 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:06:47.681790 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:06:47.681790 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:06:47.681790 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:06:47.681790 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:06:47.681790 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:06:47.681790 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:06:47.681790 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:47.689031 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104abdaa-565c-40a0-b187-566eb9cc57dd None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:47.689679 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 322/660] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 4 09:06:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:47.691258 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e64ee-4422-4986-8b98-b42e97747afd None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:47.717544 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:47.731513 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3529739-aee9-40fb-a026-59dc2817fed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:47.732677 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3529739-aee9-40fb-a026-59dc2817fed8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:47.733220 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3529739-aee9-40fb-a026-59dc2817fed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:47.733744 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3529739-aee9-40fb-a026-59dc2817fed8 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:47.734803 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3529739-aee9-40fb-a026-59dc2817fed8 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:47.735319 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3529739-aee9-40fb-a026-59dc2817fed8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:47.737197 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3529739-aee9-40fb-a026-59dc2817fed8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:47.756520 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 339/661] 10.222.0.71 () {56 vars in 1139 bytes} [Mon Jul 4 09:06:47 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:47.758509 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3529739-aee9-40fb-a026-59dc2817fed8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:47.818432 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e64ee-4422-4986-8b98-b42e97747afd None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:47.855799 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ee2ebd-69a9-4aa4-8700-79394cd14b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:47.856830 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ee2ebd-69a9-4aa4-8700-79394cd14b83 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:47.857402 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ee2ebd-69a9-4aa4-8700-79394cd14b83 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:47.857912 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ee2ebd-69a9-4aa4-8700-79394cd14b83 None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:47.859045 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ee2ebd-69a9-4aa4-8700-79394cd14b83 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:47.859573 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ee2ebd-69a9-4aa4-8700-79394cd14b83 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:47.861251 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ee2ebd-69a9-4aa4-8700-79394cd14b83 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:47.880892 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 323/662] 10.222.0.71 () {56 vars in 1137 bytes} [Mon Jul 4 09:06:47 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:47.882137 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ee2ebd-69a9-4aa4-8700-79394cd14b83 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:47.901394 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3529739-aee9-40fb-a026-59dc2817fed8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:47.945178 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0549dc90-eda5-42da-986d-87b3449e1651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:47.946781 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0549dc90-eda5-42da-986d-87b3449e1651 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:47.947274 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0549dc90-eda5-42da-986d-87b3449e1651 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:47.947779 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0549dc90-eda5-42da-986d-87b3449e1651 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:47.948878 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0549dc90-eda5-42da-986d-87b3449e1651 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:47.949415 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0549dc90-eda5-42da-986d-87b3449e1651 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:47.951048 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0549dc90-eda5-42da-986d-87b3449e1651 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:47.963549 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 340/663] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 4 09:06:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:47.969323 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0549dc90-eda5-42da-986d-87b3449e1651 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:47.977733 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:48.019714 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ee2ebd-69a9-4aa4-8700-79394cd14b83 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:48.027197 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:48.068470 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1443f5d-0b21-4446-bba6-4791137db486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:48.069313 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1443f5d-0b21-4446-bba6-4791137db486 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:48.069605 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1443f5d-0b21-4446-bba6-4791137db486 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:48.069889 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1443f5d-0b21-4446-bba6-4791137db486 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:48.081926 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0549dc90-eda5-42da-986d-87b3449e1651 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:48.118800 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d6b4bd-25e6-4dcb-873e-7dd2874d3be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:48.119837 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d6b4bd-25e6-4dcb-873e-7dd2874d3be0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:48.120352 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d6b4bd-25e6-4dcb-873e-7dd2874d3be0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:48.120843 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d6b4bd-25e6-4dcb-873e-7dd2874d3be0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:48.156903 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1443f5d-0b21-4446-bba6-4791137db486 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:48.158790 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1443f5d-0b21-4446-bba6-4791137db486 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:48.209563 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d6b4bd-25e6-4dcb-873e-7dd2874d3be0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:48.211333 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d6b4bd-25e6-4dcb-873e-7dd2874d3be0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:48.225840 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:48.240858 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:48.278190 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 324/664] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:06:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:06:48.279040 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1443f5d-0b21-4446-bba6-4791137db486 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:48.335218 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 341/665] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:06:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:06:48.335218 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db2f0fb-d660-4137-88c5-058ed71aafb4 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:48.335218 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db2f0fb-d660-4137-88c5-058ed71aafb4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:48.335218 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db2f0fb-d660-4137-88c5-058ed71aafb4 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:48.335218 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 342/666] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:48.335218 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db2f0fb-d660-4137-88c5-058ed71aafb4 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:48.395943 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1443f5d-0b21-4446-bba6-4791137db486 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:48.433242 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb81fd6-661b-4886-a268-2b6b5991f217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:48.433828 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb81fd6-661b-4886-a268-2b6b5991f217 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:48.434215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb81fd6-661b-4886-a268-2b6b5991f217 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:48.434505 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb81fd6-661b-4886-a268-2b6b5991f217 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:48.435348 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb81fd6-661b-4886-a268-2b6b5991f217 None admin] RBAC: Adding query filter params (role.id=5752a9bdfe4b482ab37836eba7bd536b, scope.project.id=4076e316243946e980ce47386b18b28f, user.id=192724527d774817bc4581474ac848dd) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:48.435668 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb81fd6-661b-4886-a268-2b6b5991f217 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:48.437155 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb81fd6-661b-4886-a268-2b6b5991f217 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:48.443686 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 325/667] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 4 09:06:48 2022] GET /identity/v3/role_assignments?user.id=192724527d774817bc4581474ac848dd&scope.project.id=4076e316243946e980ce47386b18b28f&role.id=5752a9bdfe4b482ab37836eba7bd536b => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:48.450174 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb81fd6-661b-4886-a268-2b6b5991f217 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:48.450901 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db2f0fb-d660-4137-88c5-058ed71aafb4 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:48.465379 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:48.495179 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb3471e-6e3a-49b1-a0a0-8c9f446b3a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:48.496182 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb3471e-6e3a-49b1-a0a0-8c9f446b3a9c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:48.496517 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb3471e-6e3a-49b1-a0a0-8c9f446b3a9c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:48.496896 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb3471e-6e3a-49b1-a0a0-8c9f446b3a9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:48.500819 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:48.509855 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb3471e-6e3a-49b1-a0a0-8c9f446b3a9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:48.511498 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb3471e-6e3a-49b1-a0a0-8c9f446b3a9c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:48.532270 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-afb3471e-6e3a-49b1-a0a0-8c9f446b3a9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:48.532270 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:48.532270 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:48.532270 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:48.532270 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:48.532270 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:48.532270 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:48.532270 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:48.532270 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:48.532270 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:48.532270 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:48.532270 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:48.532270 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:06:48.532270 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:06:48.532270 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:06:48.533319 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:06:48.533319 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:48.533319 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:48.533319 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:48.533319 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:48.533319 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:48.533319 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:48.533319 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:48.533319 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:48.533319 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:48.533319 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:48.533319 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:48.533319 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:48.533319 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:48.533319 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:48.533319 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:48.533319 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:48.534304 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:06:48.534304 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:06:48.534304 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:48.534304 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:48.534304 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 343/668] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:06:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:48.536336 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a2893-d101-4b0a-98be-1d8b6164040f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:48.536476 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a2893-d101-4b0a-98be-1d8b6164040f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:48.536825 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a2893-d101-4b0a-98be-1d8b6164040f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:48.557126 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0e4a2893-d101-4b0a-98be-1d8b6164040f None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:48.558503 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0e4a2893-d101-4b0a-98be-1d8b6164040f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:48.582152 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb81fd6-661b-4886-a268-2b6b5991f217 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:48.623266 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a5043b-b539-4e48-920f-82f5ed394ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:48.624409 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a5043b-b539-4e48-920f-82f5ed394ccf None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:48.624896 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a5043b-b539-4e48-920f-82f5ed394ccf None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:48.625439 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a5043b-b539-4e48-920f-82f5ed394ccf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:48.637268 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a5043b-b539-4e48-920f-82f5ed394ccf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:48.638986 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a5043b-b539-4e48-920f-82f5ed394ccf None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:48.645123 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 344/669] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:06:48.646057 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4a2893-d101-4b0a-98be-1d8b6164040f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:48.658683 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-99a5043b-b539-4e48-920f-82f5ed394ccf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:48.658683 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:48.658683 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:48.658683 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:48.658683 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:48.658683 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:48.658683 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:48.658683 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:48.658683 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:48.658683 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:48.658683 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:48.658683 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:48.658683 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:06:48.658683 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:06:48.658683 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:06:48.659636 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:06:48.659636 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:48.659636 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:48.659636 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:48.659636 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:48.659636 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:48.659636 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:48.659636 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:48.659636 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:48.659636 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:48.659636 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:48.659636 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:48.659636 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:48.659636 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:48.659636 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:48.659636 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:48.659636 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:48.660535 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:06:48.660535 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:06:48.660535 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:48.660535 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:48.662890 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 326/670] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:06:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:48.670118 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a5043b-b539-4e48-920f-82f5ed394ccf None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:48.769450 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:48.778734 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:48.787815 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4a2893-d101-4b0a-98be-1d8b6164040f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:48.792196 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a5043b-b539-4e48-920f-82f5ed394ccf None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:48.829977 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4c87cf-7fb0-4131-9c5c-85dece95a896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:48.830688 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4c87cf-7fb0-4131-9c5c-85dece95a896 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:48.831017 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4c87cf-7fb0-4131-9c5c-85dece95a896 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:48.831239 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4c87cf-7fb0-4131-9c5c-85dece95a896 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:48.831967 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4c87cf-7fb0-4131-9c5c-85dece95a896 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:48.832273 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4c87cf-7fb0-4131-9c5c-85dece95a896 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:48.833684 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4c87cf-7fb0-4131-9c5c-85dece95a896 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:48.835669 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf7bf4c-0072-4d03-b3c6-2b1a1dd27904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:48.838771 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf7bf4c-0072-4d03-b3c6-2b1a1dd27904 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:48.839272 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf7bf4c-0072-4d03-b3c6-2b1a1dd27904 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:48.839810 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf7bf4c-0072-4d03-b3c6-2b1a1dd27904 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:48.847388 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 345/671] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:06:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:48.848221 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4c87cf-7fb0-4131-9c5c-85dece95a896 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:48.856546 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf7bf4c-0072-4d03-b3c6-2b1a1dd27904 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:06:48.866638 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-cbf7bf4c-0072-4d03-b3c6-2b1a1dd27904 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:06:48.866638 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:48.866638 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:48.866638 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:48.866638 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:48.866638 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:48.866638 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:48.866638 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:48.866638 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:48.866638 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:48.866638 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:48.866638 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:48.866638 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:06:48.866638 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:06:48.866638 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:06:48.866638 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:06:48.867857 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:06:48.867857 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:06:48.867857 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:06:48.867857 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:06:48.867857 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:48.867857 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:48.867857 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:48.867857 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:48.867857 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:48.867857 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:48.867857 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:48.867857 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:48.867857 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:48.867857 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:48.867857 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:48.867857 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:48.868812 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:48.868812 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:48.868812 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:48.868812 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:48.868812 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:06:48.868812 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:06:48.868812 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:06:48.868812 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:06:48.868812 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:06:48.868812 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:06:48.868812 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:06:48.868812 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:48.868812 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 327/672] 10.222.0.71 () {56 vars in 1102 bytes} [Mon Jul 4 09:06:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:48.870496 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf7bf4c-0072-4d03-b3c6-2b1a1dd27904 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:48.965470 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4c87cf-7fb0-4131-9c5c-85dece95a896 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:48.983160 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf7bf4c-0072-4d03-b3c6-2b1a1dd27904 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:49.002455 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae77257c-9441-499f-ab0b-70706109c492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:49.003122 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae77257c-9441-499f-ab0b-70706109c492 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:49.003402 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae77257c-9441-499f-ab0b-70706109c492 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:49.004098 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae77257c-9441-499f-ab0b-70706109c492 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:49.005114 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae77257c-9441-499f-ab0b-70706109c492 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:49.005430 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae77257c-9441-499f-ab0b-70706109c492 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:49.007266 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae77257c-9441-499f-ab0b-70706109c492 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:49.019192 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9122f6d-9091-42e3-9759-35912f1dbead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:49.020204 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9122f6d-9091-42e3-9759-35912f1dbead None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:49.020442 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9122f6d-9091-42e3-9759-35912f1dbead None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:49.020723 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9122f6d-9091-42e3-9759-35912f1dbead None admin] PATH_INFO: `/v3/projects/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:49.026628 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 346/673] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:06:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:49.026628 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae77257c-9441-499f-ab0b-70706109c492 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:49.028720 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9122f6d-9091-42e3-9759-35912f1dbead None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:49.031012 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9122f6d-9091-42e3-9759-35912f1dbead None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:49.042939 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:49.044917 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-d9122f6d-9091-42e3-9759-35912f1dbead None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:06:49.044917 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:49.044917 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:49.044917 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:49.044917 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:49.044917 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:49.044917 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:49.044917 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:49.044917 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:49.044917 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:49.044917 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:49.044917 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:49.044917 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:06:49.044917 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:06:49.044917 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:06:49.045963 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:06:49.045963 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:49.045963 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:49.045963 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:49.045963 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:49.045963 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:49.045963 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:49.045963 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:49.045963 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:49.045963 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:49.045963 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:49.045963 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:49.045963 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:49.045963 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:49.045963 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:49.045963 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:49.045963 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:49.046883 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:06:49.046883 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:06:49.046883 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:06:49.046883 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:06:49.046883 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:06:49.046883 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:06:49.046883 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:06:49.046883 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:49.046883 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 328/674] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 4 09:06:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:49.047371 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b106d3ff-3e83-44a6-9cb1-3dba93d3642e None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:49.048075 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b106d3ff-3e83-44a6-9cb1-3dba93d3642e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:49.048075 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b106d3ff-3e83-44a6-9cb1-3dba93d3642e None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:49.049454 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 329/675] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:49.050135 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b106d3ff-3e83-44a6-9cb1-3dba93d3642e None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:49.063008 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:49.141475 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae77257c-9441-499f-ab0b-70706109c492 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:49.177875 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8719b955-9ecd-4381-b4ef-c66ce17c8ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:49.178554 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8719b955-9ecd-4381-b4ef-c66ce17c8ad0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:49.178846 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8719b955-9ecd-4381-b4ef-c66ce17c8ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:49.179122 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8719b955-9ecd-4381-b4ef-c66ce17c8ad0 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:49.179905 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8719b955-9ecd-4381-b4ef-c66ce17c8ad0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:49.179905 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8719b955-9ecd-4381-b4ef-c66ce17c8ad0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:49.182046 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8719b955-9ecd-4381-b4ef-c66ce17c8ad0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:49.184505 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b106d3ff-3e83-44a6-9cb1-3dba93d3642e None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:49.193484 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 347/676] 10.222.0.71 () {56 vars in 1103 bytes} [Mon Jul 4 09:06:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:49.194472 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8719b955-9ecd-4381-b4ef-c66ce17c8ad0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:49.226502 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a90783-2165-44cf-ad5d-60fce63cd40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:49.227772 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a90783-2165-44cf-ad5d-60fce63cd40d None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:49.228052 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a90783-2165-44cf-ad5d-60fce63cd40d None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:49.228327 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a90783-2165-44cf-ad5d-60fce63cd40d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:49.235223 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a90783-2165-44cf-ad5d-60fce63cd40d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:49.236851 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a90783-2165-44cf-ad5d-60fce63cd40d None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:49.250986 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-18a90783-2165-44cf-ad5d-60fce63cd40d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:06:49.250986 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:49.250986 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:49.250986 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:49.250986 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:49.250986 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:49.250986 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:49.250986 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:49.250986 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:49.250986 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:49.250986 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:49.250986 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:49.250986 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:06:49.250986 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:06:49.250986 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:06:49.251919 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:06:49.251919 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:49.251919 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:49.251919 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:49.251919 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:49.251919 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:49.251919 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:49.251919 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:49.251919 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:49.251919 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:49.251919 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:49.251919 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:49.251919 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:49.251919 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:49.251919 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:49.251919 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:49.251919 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:49.252836 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:06:49.252836 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:06:49.252836 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:06:49.252836 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:06:49.252836 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:06:49.252836 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:06:49.252836 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:06:49.252836 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:49.252836 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 330/677] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 4 09:06:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:49.254179 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b963a2d-7cd8-45fe-8c52-a70ae2f06bbc None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:49.254179 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b963a2d-7cd8-45fe-8c52-a70ae2f06bbc None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:49.254179 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b963a2d-7cd8-45fe-8c52-a70ae2f06bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:49.272090 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-1b963a2d-7cd8-45fe-8c52-a70ae2f06bbc None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:49.273263 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-1b963a2d-7cd8-45fe-8c52-a70ae2f06bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:49.286840 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:49.314167 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8719b955-9ecd-4381-b4ef-c66ce17c8ad0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:49.344576 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:49.351045 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 331/678] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:06:49.352195 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b963a2d-7cd8-45fe-8c52-a70ae2f06bbc None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:49.354944 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28c5e37-a107-4e3c-934d-ca2ffe168556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:49.356795 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28c5e37-a107-4e3c-934d-ca2ffe168556 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:49.357343 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28c5e37-a107-4e3c-934d-ca2ffe168556 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:49.357857 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28c5e37-a107-4e3c-934d-ca2ffe168556 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:49.358916 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28c5e37-a107-4e3c-934d-ca2ffe168556 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:49.359446 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28c5e37-a107-4e3c-934d-ca2ffe168556 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:49.361124 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28c5e37-a107-4e3c-934d-ca2ffe168556 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:49.380660 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 348/679] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 4 09:06:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:49.381964 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28c5e37-a107-4e3c-934d-ca2ffe168556 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:49.485427 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b963a2d-7cd8-45fe-8c52-a70ae2f06bbc None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:49.501134 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28c5e37-a107-4e3c-934d-ca2ffe168556 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:49.530691 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e592546b-6650-4b70-bf0e-0efacc212c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:49.531397 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e592546b-6650-4b70-bf0e-0efacc212c21 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:49.531678 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e592546b-6650-4b70-bf0e-0efacc212c21 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:49.531953 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e592546b-6650-4b70-bf0e-0efacc212c21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:49.544868 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e592546b-6650-4b70-bf0e-0efacc212c21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:49.546835 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e592546b-6650-4b70-bf0e-0efacc212c21 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:49.560414 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd55af9-d54f-4b33-8cbb-9ed480a561f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:49.561022 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd55af9-d54f-4b33-8cbb-9ed480a561f6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:49.561241 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd55af9-d54f-4b33-8cbb-9ed480a561f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:49.561524 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd55af9-d54f-4b33-8cbb-9ed480a561f6 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:49.562322 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd55af9-d54f-4b33-8cbb-9ed480a561f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:49.562642 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd55af9-d54f-4b33-8cbb-9ed480a561f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:49.564064 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd55af9-d54f-4b33-8cbb-9ed480a561f6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:49.565743 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:49.574815 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-e592546b-6650-4b70-bf0e-0efacc212c21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:49.574815 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:49.574815 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:49.574815 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:49.574815 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:49.574815 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:49.574815 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:49.574815 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:49.574815 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:49.574815 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:49.574815 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:49.574815 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:49.574815 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:06:49.574815 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:06:49.574815 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:06:49.575769 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:06:49.575769 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:49.575769 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:49.575769 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:49.575769 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:49.575769 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:49.575769 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:49.575769 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:49.575769 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:49.575769 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:49.575769 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:49.575769 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:49.575769 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:49.575769 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:49.575769 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:49.575769 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:49.575769 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:49.576666 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:06:49.576666 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:06:49.576666 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:49.576666 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:49.577208 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 332/680] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:06:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:49.578285 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e592546b-6650-4b70-bf0e-0efacc212c21 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:49.584281 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 349/681] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 4 09:06:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:49.585621 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd55af9-d54f-4b33-8cbb-9ed480a561f6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:49.643416 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:49.710284 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e592546b-6650-4b70-bf0e-0efacc212c21 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:49.722731 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd55af9-d54f-4b33-8cbb-9ed480a561f6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:49.752853 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cb6396-b502-44fd-b1ee-5829f3295df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:49.753817 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb6396-b502-44fd-b1ee-5829f3295df6 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:49.754110 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb6396-b502-44fd-b1ee-5829f3295df6 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:49.754382 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb6396-b502-44fd-b1ee-5829f3295df6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:49.763241 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ca189-caa9-4e2c-9fb6-ae00b2e7101d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:49.764334 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ca189-caa9-4e2c-9fb6-ae00b2e7101d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:49.764774 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ca189-caa9-4e2c-9fb6-ae00b2e7101d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:49.765234 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ca189-caa9-4e2c-9fb6-ae00b2e7101d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:49.766329 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3ca189-caa9-4e2c-9fb6-ae00b2e7101d None admin] RBAC: Adding query filter params (role.id=5752a9bdfe4b482ab37836eba7bd536b, scope.project.id=4076e316243946e980ce47386b18b28f, user.id=15818e58661646f381da0973d4c2ff64) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:49.766798 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3ca189-caa9-4e2c-9fb6-ae00b2e7101d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:49.767637 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cb6396-b502-44fd-b1ee-5829f3295df6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:06:49.768543 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3ca189-caa9-4e2c-9fb6-ae00b2e7101d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:49.775651 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 350/682] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 4 09:06:49 2022] GET /identity/v3/role_assignments?user.id=15818e58661646f381da0973d4c2ff64&scope.project.id=4076e316243946e980ce47386b18b28f&role.id=5752a9bdfe4b482ab37836eba7bd536b => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:49.776755 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ca189-caa9-4e2c-9fb6-ae00b2e7101d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:49.780842 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-08cb6396-b502-44fd-b1ee-5829f3295df6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:06:49.780842 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:49.780842 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:49.780842 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:49.780842 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:49.780842 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:49.780842 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:49.780842 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:49.780842 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:49.780842 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:49.780842 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:49.780842 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:49.780842 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:06:49.780842 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:06:49.780842 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:06:49.780842 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:06:49.782084 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:06:49.782084 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:06:49.782084 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:06:49.782084 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:06:49.782084 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:49.782084 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:49.782084 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:49.782084 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:49.782084 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:49.782084 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:49.782084 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:49.782084 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:49.782084 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:49.782084 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:49.782084 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:49.782084 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:49.783125 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:49.783125 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:49.783125 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:49.783125 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:49.783125 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:06:49.783125 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:06:49.783125 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:06:49.783125 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:06:49.783125 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:06:49.783125 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:06:49.783125 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:06:49.783125 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:49.783125 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 333/683] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 4 09:06:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:49.783842 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cb6396-b502-44fd-b1ee-5829f3295df6 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:49.886542 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:49.950066 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ca189-caa9-4e2c-9fb6-ae00b2e7101d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:49.962531 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cb6396-b502-44fd-b1ee-5829f3295df6 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:49.975892 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:50.001975 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b2a24e-5d02-4112-a5ec-cda357bb24c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:50.003425 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b2a24e-5d02-4112-a5ec-cda357bb24c9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:50.004964 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b2a24e-5d02-4112-a5ec-cda357bb24c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:50.005416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b2a24e-5d02-4112-a5ec-cda357bb24c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:50.006334 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b2a24e-5d02-4112-a5ec-cda357bb24c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:50.006833 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b2a24e-5d02-4112-a5ec-cda357bb24c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:50.008956 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b2a24e-5d02-4112-a5ec-cda357bb24c9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:50.011285 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77485458-085b-4bdf-b0d7-70b3af8008d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:50.012418 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77485458-085b-4bdf-b0d7-70b3af8008d2 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:50.012707 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77485458-085b-4bdf-b0d7-70b3af8008d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:50.012991 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77485458-085b-4bdf-b0d7-70b3af8008d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:50.013883 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77485458-085b-4bdf-b0d7-70b3af8008d2 None admin] RBAC: Adding query filter params (role.id=5752a9bdfe4b482ab37836eba7bd536b, scope.project.id=4076e316243946e980ce47386b18b28f, user.id=f6e50a3715a14714b3951d36338dde42) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:50.014204 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77485458-085b-4bdf-b0d7-70b3af8008d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:50.015643 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77485458-085b-4bdf-b0d7-70b3af8008d2 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:50.024361 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 334/684] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 4 09:06:49 2022] GET /identity/v3/role_assignments?user.id=f6e50a3715a14714b3951d36338dde42&scope.project.id=4076e316243946e980ce47386b18b28f&role.id=5752a9bdfe4b482ab37836eba7bd536b => generated 608 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:50.025513 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77485458-085b-4bdf-b0d7-70b3af8008d2 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:50.026384 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 351/685] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:06:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:50.035300 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b2a24e-5d02-4112-a5ec-cda357bb24c9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:50.172344 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77485458-085b-4bdf-b0d7-70b3af8008d2 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:50.189222 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b2a24e-5d02-4112-a5ec-cda357bb24c9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:50.213049 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b91f56f-2323-4cd2-bd13-27934717f3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:50.213812 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b91f56f-2323-4cd2-bd13-27934717f3dd None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:50.214109 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b91f56f-2323-4cd2-bd13-27934717f3dd None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:50.214396 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b91f56f-2323-4cd2-bd13-27934717f3dd None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:50.215166 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b91f56f-2323-4cd2-bd13-27934717f3dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:50.215488 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b91f56f-2323-4cd2-bd13-27934717f3dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:50.215614 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:50.217462 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b91f56f-2323-4cd2-bd13-27934717f3dd None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:50.228406 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 335/686] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:06:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:50.234949 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b91f56f-2323-4cd2-bd13-27934717f3dd None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:50.238302 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf170b7-0fed-4e6b-bb60-44a13990b6a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:50.239127 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf170b7-0fed-4e6b-bb60-44a13990b6a8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:50.239413 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf170b7-0fed-4e6b-bb60-44a13990b6a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:50.239677 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf170b7-0fed-4e6b-bb60-44a13990b6a8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:50.248409 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf170b7-0fed-4e6b-bb60-44a13990b6a8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:50.251269 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf170b7-0fed-4e6b-bb60-44a13990b6a8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:50.273495 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-daf170b7-0fed-4e6b-bb60-44a13990b6a8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 04 09:06:50.273495 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:50.273495 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:50.273495 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:50.273495 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:50.273495 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:50.273495 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:50.273495 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:50.273495 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:50.273495 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:50.273495 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:50.273495 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:50.273495 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:06:50.273495 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:06:50.273495 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:06:50.273495 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:06:50.275253 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:50.275253 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:50.275253 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:06:50.275253 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:06:50.275253 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:06:50.275253 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:06:50.275253 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:50.275253 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:50.275253 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:50.275253 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:50.275253 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:50.275253 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:50.275253 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:50.275253 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:50.275253 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:50.275253 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:50.275253 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:50.275253 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:50.276232 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:50.276232 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:50.276232 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:06:50.276232 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:06:50.276232 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:06:50.276232 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:06:50.276232 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:06:50.276232 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:06:50.276232 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 04 09:06:50.276232 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:50.276232 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 352/687] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 4 09:06:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:06:50.278810 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:50.282654 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf170b7-0fed-4e6b-bb60-44a13990b6a8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:50.362107 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b91f56f-2323-4cd2-bd13-27934717f3dd None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:50.400533 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f70a41-0b43-44e4-be09-0e32c6061fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:50.401387 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f70a41-0b43-44e4-be09-0e32c6061fea None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:50.401677 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f70a41-0b43-44e4-be09-0e32c6061fea None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:50.401959 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f70a41-0b43-44e4-be09-0e32c6061fea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:50.411938 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf170b7-0fed-4e6b-bb60-44a13990b6a8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:50.413704 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f70a41-0b43-44e4-be09-0e32c6061fea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:50.416133 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f70a41-0b43-44e4-be09-0e32c6061fea None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:50.435675 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-94f70a41-0b43-44e4-be09-0e32c6061fea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:50.435675 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:50.435675 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:50.435675 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:50.435675 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:50.435675 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:50.435675 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:50.435675 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:50.435675 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:50.435675 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:50.435675 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:50.435675 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:50.435675 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:06:50.435675 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:06:50.435675 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:06:50.436650 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:06:50.436650 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:50.436650 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:50.436650 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:50.436650 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:50.436650 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:50.436650 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:50.436650 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:50.436650 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:50.436650 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:50.436650 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:50.436650 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:50.436650 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:50.436650 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:50.436650 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:50.436650 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:50.436650 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:50.438020 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:06:50.438020 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:06:50.438020 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:50.438020 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:50.438020 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 336/688] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:06:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:50.443833 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f70a41-0b43-44e4-be09-0e32c6061fea None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:50.455781 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a37458-aa49-4de5-a0c4-c076ac5ef76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:50.456797 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a37458-aa49-4de5-a0c4-c076ac5ef76d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:50.457398 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a37458-aa49-4de5-a0c4-c076ac5ef76d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:50.457878 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a37458-aa49-4de5-a0c4-c076ac5ef76d None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:50.458938 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a37458-aa49-4de5-a0c4-c076ac5ef76d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:50.459534 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a37458-aa49-4de5-a0c4-c076ac5ef76d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:50.461824 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a37458-aa49-4de5-a0c4-c076ac5ef76d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:50.482479 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 353/689] 10.222.0.71 () {56 vars in 1135 bytes} [Mon Jul 4 09:06:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:06:50.487439 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a37458-aa49-4de5-a0c4-c076ac5ef76d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:50.526843 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:50.545229 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:50.559423 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f70a41-0b43-44e4-be09-0e32c6061fea None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:50.597202 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1434d8-c431-4dd7-baf4-4dac2dcccece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:50.597882 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1434d8-c431-4dd7-baf4-4dac2dcccece None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:50.598173 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1434d8-c431-4dd7-baf4-4dac2dcccece None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:50.598455 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1434d8-c431-4dd7-baf4-4dac2dcccece None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:50.599203 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1434d8-c431-4dd7-baf4-4dac2dcccece None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:50.599675 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1434d8-c431-4dd7-baf4-4dac2dcccece None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:50.600891 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1434d8-c431-4dd7-baf4-4dac2dcccece None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:50.606638 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a37458-aa49-4de5-a0c4-c076ac5ef76d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:50.615675 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 337/690] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:06:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:50.621676 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1434d8-c431-4dd7-baf4-4dac2dcccece None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:50.645773 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071af49e-c085-45f1-b371-537ccfbe6a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:50.646820 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071af49e-c085-45f1-b371-537ccfbe6a68 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:50.647317 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071af49e-c085-45f1-b371-537ccfbe6a68 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:50.647807 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071af49e-c085-45f1-b371-537ccfbe6a68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:50.725423 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071af49e-c085-45f1-b371-537ccfbe6a68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:50.728167 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071af49e-c085-45f1-b371-537ccfbe6a68 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:50.732170 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1434d8-c431-4dd7-baf4-4dac2dcccece None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:50.767596 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:50.768636 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535c60e-4b87-4b9f-afcd-1916827dcacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:50.769653 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535c60e-4b87-4b9f-afcd-1916827dcacc None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:50.769768 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535c60e-4b87-4b9f-afcd-1916827dcacc None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:50.770009 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535c60e-4b87-4b9f-afcd-1916827dcacc None admin] PATH_INFO: `/v3/users/nova` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:50.778928 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1535c60e-4b87-4b9f-afcd-1916827dcacc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:50.780864 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1535c60e-4b87-4b9f-afcd-1916827dcacc None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:50.799855 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-1535c60e-4b87-4b9f-afcd-1916827dcacc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 04 09:06:50.799855 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:50.799855 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:50.799855 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:50.799855 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:50.799855 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:50.799855 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:50.799855 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:50.799855 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:50.799855 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:50.799855 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:50.799855 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:50.799855 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:06:50.799855 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:06:50.799855 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:06:50.799855 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:06:50.800952 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:50.800952 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:50.800952 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:06:50.800952 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:06:50.800952 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:06:50.800952 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:06:50.800952 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:50.800952 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:50.800952 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:50.800952 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:50.800952 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:50.800952 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:50.800952 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:50.800952 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:50.800952 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:50.800952 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:50.800952 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:50.800952 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:50.802699 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:50.802699 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:50.802699 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:06:50.802699 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:06:50.802699 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:06:50.802699 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:06:50.802699 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:06:50.802699 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:06:50.802699 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 04 09:06:50.802699 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:50.802699 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 338/691] 10.222.0.71 () {56 vars in 1093 bytes} [Mon Jul 4 09:06:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:06:50.808180 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535c60e-4b87-4b9f-afcd-1916827dcacc None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:50.819761 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:50.855964 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 354/692] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:06:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:06:50.857728 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071af49e-c085-45f1-b371-537ccfbe6a68 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:50.921068 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535c60e-4b87-4b9f-afcd-1916827dcacc None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:50.958786 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd7dd73-867b-4bc7-850a-9d213feda0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:50.959644 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd7dd73-867b-4bc7-850a-9d213feda0a7 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:50.959850 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd7dd73-867b-4bc7-850a-9d213feda0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:50.960136 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd7dd73-867b-4bc7-850a-9d213feda0a7 None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:50.961943 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd7dd73-867b-4bc7-850a-9d213feda0a7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:50.962515 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd7dd73-867b-4bc7-850a-9d213feda0a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:50.964092 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd7dd73-867b-4bc7-850a-9d213feda0a7 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:50.968060 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071af49e-c085-45f1-b371-537ccfbe6a68 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:50.984128 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 339/693] 10.222.0.71 () {56 vars in 1133 bytes} [Mon Jul 4 09:06:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:50.989958 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e5ef59-5ed1-434a-83bf-c670a7125381 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:50.990259 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e5ef59-5ed1-434a-83bf-c670a7125381 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:50.990550 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e5ef59-5ed1-434a-83bf-c670a7125381 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:50.992660 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 340/694] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:50 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:50.993784 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e5ef59-5ed1-434a-83bf-c670a7125381 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:51.009217 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480f86ae-e115-401e-8873-f4ed52d894fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:51.010791 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f86ae-e115-401e-8873-f4ed52d894fb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:51.011301 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f86ae-e115-401e-8873-f4ed52d894fb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:51.011832 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f86ae-e115-401e-8873-f4ed52d894fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:51.015077 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:51.024114 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480f86ae-e115-401e-8873-f4ed52d894fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:51.026088 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480f86ae-e115-401e-8873-f4ed52d894fb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:51.044986 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-480f86ae-e115-401e-8873-f4ed52d894fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:51.044986 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:51.044986 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:51.044986 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:51.044986 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:51.044986 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:51.044986 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:51.044986 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:51.044986 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:51.044986 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:51.044986 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:51.044986 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:51.044986 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:06:51.044986 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:06:51.044986 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:06:51.045931 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:06:51.045931 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:51.045931 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:51.045931 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:51.045931 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:51.045931 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:51.045931 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:51.045931 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:51.045931 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:51.045931 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:51.045931 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:51.045931 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:51.045931 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:51.045931 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:51.045931 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:51.045931 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:51.045931 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:51.046840 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:06:51.046840 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:06:51.046840 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:51.046840 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:51.053474 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 355/695] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:06:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:51.055678 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c0a10-0b63-44d3-8fac-1505ad040331 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:51.056199 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c0a10-0b63-44d3-8fac-1505ad040331 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:51.056674 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c0a10-0b63-44d3-8fac-1505ad040331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:51.073423 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:51.079616 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-465c0a10-0b63-44d3-8fac-1505ad040331 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:51.081064 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-465c0a10-0b63-44d3-8fac-1505ad040331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:51.115327 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e5ef59-5ed1-434a-83bf-c670a7125381 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:51.155064 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 356/696] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:06:51.156355 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c0a10-0b63-44d3-8fac-1505ad040331 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:51.157590 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6b8728-4003-4b70-b5e4-b71cb8d65cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:51.158316 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6b8728-4003-4b70-b5e4-b71cb8d65cb9 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:51.158914 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6b8728-4003-4b70-b5e4-b71cb8d65cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:51.159550 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6b8728-4003-4b70-b5e4-b71cb8d65cb9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:51.238326 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6b8728-4003-4b70-b5e4-b71cb8d65cb9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:51.240003 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6b8728-4003-4b70-b5e4-b71cb8d65cb9 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:51.259552 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c0a10-0b63-44d3-8fac-1505ad040331 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:51.278649 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:51.297989 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8c7996-e3f9-4d67-a0cd-94827faeccd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:51.298988 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8c7996-e3f9-4d67-a0cd-94827faeccd1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:51.299464 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8c7996-e3f9-4d67-a0cd-94827faeccd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:51.299885 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8c7996-e3f9-4d67-a0cd-94827faeccd1 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:51.300799 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8c7996-e3f9-4d67-a0cd-94827faeccd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:51.301302 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8c7996-e3f9-4d67-a0cd-94827faeccd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:51.302729 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8c7996-e3f9-4d67-a0cd-94827faeccd1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:51.321492 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 357/697] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:06:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:51.322481 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8c7996-e3f9-4d67-a0cd-94827faeccd1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:51.332599 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:51.388625 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 341/698] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:06:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:06:51.389699 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6b8728-4003-4b70-b5e4-b71cb8d65cb9 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:51.446370 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8c7996-e3f9-4d67-a0cd-94827faeccd1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:51.482940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2574d3-4c8d-4650-b72a-c261466bb78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:51.484000 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2574d3-4c8d-4650-b72a-c261466bb78d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:51.484445 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2574d3-4c8d-4650-b72a-c261466bb78d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:51.484869 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2574d3-4c8d-4650-b72a-c261466bb78d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:51.496792 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2574d3-4c8d-4650-b72a-c261466bb78d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:51.497855 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6b8728-4003-4b70-b5e4-b71cb8d65cb9 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:51.498635 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2574d3-4c8d-4650-b72a-c261466bb78d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:51.517325 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-0e2574d3-4c8d-4650-b72a-c261466bb78d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:51.517325 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:51.517325 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:51.517325 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:51.517325 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:51.517325 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:51.517325 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:51.517325 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:51.517325 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:51.517325 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:51.517325 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:51.517325 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:51.517325 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:06:51.517325 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:06:51.517325 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:06:51.518411 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:06:51.518411 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:51.518411 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:51.518411 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:51.518411 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:51.518411 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:51.518411 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:51.518411 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:51.518411 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:51.518411 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:51.518411 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:51.518411 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:51.518411 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:51.518411 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:51.518411 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:51.518411 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:51.518411 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:51.519532 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:06:51.519532 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:06:51.519532 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:51.519532 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:51.526441 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 358/699] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:06:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:51.527597 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2574d3-4c8d-4650-b72a-c261466bb78d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:51.534069 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd97b8a-261f-4e33-b428-5d40971f657d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:51.534877 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd97b8a-261f-4e33-b428-5d40971f657d None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:51.535159 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd97b8a-261f-4e33-b428-5d40971f657d None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:51.535432 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd97b8a-261f-4e33-b428-5d40971f657d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:51.542755 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd97b8a-261f-4e33-b428-5d40971f657d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:51.544392 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd97b8a-261f-4e33-b428-5d40971f657d None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:51.559741 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-5cd97b8a-261f-4e33-b428-5d40971f657d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:06:51.559741 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:51.559741 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:51.559741 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:51.559741 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:51.559741 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:51.559741 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:51.559741 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:51.559741 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:51.559741 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:51.559741 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:51.559741 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:51.559741 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:06:51.559741 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:06:51.559741 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:06:51.560826 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:06:51.560826 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:51.560826 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:51.560826 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:51.560826 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:51.560826 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:51.560826 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:51.560826 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:51.560826 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:51.560826 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:51.560826 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:51.560826 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:51.560826 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:51.560826 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:51.560826 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:51.560826 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:51.560826 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:51.561970 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:06:51.561970 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:06:51.561970 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:06:51.561970 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:06:51.561970 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:06:51.561970 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:06:51.561970 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:06:51.561970 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:51.565598 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 342/700] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 4 09:06:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:51.566424 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd97b8a-261f-4e33-b428-5d40971f657d None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:51.567301 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:51.606583 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:51.642364 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2574d3-4c8d-4650-b72a-c261466bb78d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:51.683541 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c0925f-e387-49f9-9d63-e246309d5cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:51.684730 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c0925f-e387-49f9-9d63-e246309d5cc3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:51.685283 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c0925f-e387-49f9-9d63-e246309d5cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:51.685780 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c0925f-e387-49f9-9d63-e246309d5cc3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:51.697920 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c0925f-e387-49f9-9d63-e246309d5cc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:51.699720 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c0925f-e387-49f9-9d63-e246309d5cc3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:51.701345 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd97b8a-261f-4e33-b428-5d40971f657d None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:51.719617 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-14c0925f-e387-49f9-9d63-e246309d5cc3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:51.719617 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:51.719617 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:51.719617 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:51.719617 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:51.719617 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:51.719617 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:51.719617 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:51.719617 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:51.719617 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:51.719617 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:51.719617 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:51.719617 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:06:51.719617 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:06:51.719617 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:06:51.720657 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:06:51.720657 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:51.720657 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:51.720657 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:51.720657 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:51.720657 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:51.720657 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:51.720657 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:51.720657 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:51.720657 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:51.720657 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:51.720657 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:51.720657 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:51.720657 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:51.720657 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:51.720657 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:51.720657 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:51.721606 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:06:51.721606 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:06:51.721606 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:51.721606 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:51.721606 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 359/701] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:06:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:51.722322 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c0925f-e387-49f9-9d63-e246309d5cc3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:51.755017 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9223b4c7-12aa-4ad5-ad59-f5e36dabbbca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:51.755774 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9223b4c7-12aa-4ad5-ad59-f5e36dabbbca None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:51.756434 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9223b4c7-12aa-4ad5-ad59-f5e36dabbbca None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:51.756717 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9223b4c7-12aa-4ad5-ad59-f5e36dabbbca None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:51.758322 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9223b4c7-12aa-4ad5-ad59-f5e36dabbbca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:51.758637 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9223b4c7-12aa-4ad5-ad59-f5e36dabbbca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:51.760489 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9223b4c7-12aa-4ad5-ad59-f5e36dabbbca None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:51.777967 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 343/702] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:06:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:51.779062 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9223b4c7-12aa-4ad5-ad59-f5e36dabbbca None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:51.821168 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:51.842402 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c0925f-e387-49f9-9d63-e246309d5cc3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:51.881507 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad6e04d-3ed6-4a8f-b2c9-28919b737d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:51.882260 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad6e04d-3ed6-4a8f-b2c9-28919b737d33 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:51.882666 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad6e04d-3ed6-4a8f-b2c9-28919b737d33 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:51.883505 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad6e04d-3ed6-4a8f-b2c9-28919b737d33 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:51.884451 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad6e04d-3ed6-4a8f-b2c9-28919b737d33 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:51.885000 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad6e04d-3ed6-4a8f-b2c9-28919b737d33 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:51.886511 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad6e04d-3ed6-4a8f-b2c9-28919b737d33 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:51.899880 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 360/703] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 4 09:06:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:51.905363 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:51.906459 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad6e04d-3ed6-4a8f-b2c9-28919b737d33 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:51.929594 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9223b4c7-12aa-4ad5-ad59-f5e36dabbbca None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:51.972205 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92442ef4-1382-4c6a-ac42-99861eb11414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:51.972987 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92442ef4-1382-4c6a-ac42-99861eb11414 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:51.973305 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92442ef4-1382-4c6a-ac42-99861eb11414 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:51.973582 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92442ef4-1382-4c6a-ac42-99861eb11414 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:51.974308 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92442ef4-1382-4c6a-ac42-99861eb11414 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:51.974614 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92442ef4-1382-4c6a-ac42-99861eb11414 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:51.976097 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92442ef4-1382-4c6a-ac42-99861eb11414 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:51.989519 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 344/704] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:06:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:51.990435 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92442ef4-1382-4c6a-ac42-99861eb11414 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:52.040674 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad6e04d-3ed6-4a8f-b2c9-28919b737d33 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:52.084286 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac44dd-e753-4236-8917-2a3e90dc349a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:52.085254 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac44dd-e753-4236-8917-2a3e90dc349a None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:52.085542 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac44dd-e753-4236-8917-2a3e90dc349a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:52.086522 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac44dd-e753-4236-8917-2a3e90dc349a None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:52.087849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ac44dd-e753-4236-8917-2a3e90dc349a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:52.089627 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ac44dd-e753-4236-8917-2a3e90dc349a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:52.103891 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92442ef4-1382-4c6a-ac42-99861eb11414 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:52.109112 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:52.116551 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 361/705] 10.222.0.71 () {60 vars in 1132 bytes} [Mon Jul 4 09:06:51 2022] POST /identity/v3/users => generated 254 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:06:52.118205 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac44dd-e753-4236-8917-2a3e90dc349a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:52.149576 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02242fa8-bf1c-428f-9be8-c18fff178885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:52.150652 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02242fa8-bf1c-428f-9be8-c18fff178885 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:52.151034 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02242fa8-bf1c-428f-9be8-c18fff178885 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:52.151411 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02242fa8-bf1c-428f-9be8-c18fff178885 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:52.152598 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02242fa8-bf1c-428f-9be8-c18fff178885 None admin] RBAC: Adding query filter params (role.id=5752a9bdfe4b482ab37836eba7bd536b, scope.project.id=4076e316243946e980ce47386b18b28f, user.id=88b41b55769a4c449716c3590ed58225) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:52.153066 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02242fa8-bf1c-428f-9be8-c18fff178885 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:52.154855 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02242fa8-bf1c-428f-9be8-c18fff178885 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:52.162604 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 345/706] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 4 09:06:51 2022] GET /identity/v3/role_assignments?user.id=88b41b55769a4c449716c3590ed58225&scope.project.id=4076e316243946e980ce47386b18b28f&role.id=5752a9bdfe4b482ab37836eba7bd536b => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:52.283468 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac44dd-e753-4236-8917-2a3e90dc349a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:52.318965 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be106f1-9da9-4b69-b493-551ae2c02417 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:52.319163 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be106f1-9da9-4b69-b493-551ae2c02417 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:52.319434 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be106f1-9da9-4b69-b493-551ae2c02417 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:52.322404 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 346/707] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:06:52.328090 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eced5f-8dce-4b21-9cc6-6e9d57bd5f2f None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:52.328411 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eced5f-8dce-4b21-9cc6-6e9d57bd5f2f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:52.328652 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eced5f-8dce-4b21-9cc6-6e9d57bd5f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:52.345249 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e340b0-b1f7-49f4-b43a-9ce781824ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:52.346081 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e340b0-b1f7-49f4-b43a-9ce781824ddb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:52.346364 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e340b0-b1f7-49f4-b43a-9ce781824ddb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:52.346643 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e340b0-b1f7-49f4-b43a-9ce781824ddb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:52.359615 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-32eced5f-8dce-4b21-9cc6-6e9d57bd5f2f None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:52.360873 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-32eced5f-8dce-4b21-9cc6-6e9d57bd5f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:52.364301 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e340b0-b1f7-49f4-b43a-9ce781824ddb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:52.365840 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e340b0-b1f7-49f4-b43a-9ce781824ddb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:52.376129 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:52.390526 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-97e340b0-b1f7-49f4-b43a-9ce781824ddb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:06:52.390526 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:52.390526 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:52.390526 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:52.390526 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:52.390526 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:52.390526 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:52.390526 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:52.390526 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:52.390526 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:52.390526 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:52.390526 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:52.390526 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:06:52.390526 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:06:52.390526 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:06:52.391564 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:06:52.391564 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:52.391564 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:52.391564 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:52.391564 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:52.391564 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:52.391564 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:52.391564 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:52.391564 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:52.391564 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:52.391564 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:52.391564 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:52.391564 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:52.391564 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:52.391564 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:52.391564 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:52.391564 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:52.392498 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:06:52.392498 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:06:52.392498 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:06:52.392498 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:06:52.392498 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:06:52.392498 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:06:52.392498 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:06:52.392498 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:52.392498 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 362/708] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 4 09:06:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:52.401015 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e340b0-b1f7-49f4-b43a-9ce781824ddb None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:52.479961 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 347/709] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:06:52.481911 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:52.503598 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eced5f-8dce-4b21-9cc6-6e9d57bd5f2f None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:52.528957 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e340b0-b1f7-49f4-b43a-9ce781824ddb None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:52.572483 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72332cdb-13fc-4fec-988a-f92a54fb3d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:52.573388 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72332cdb-13fc-4fec-988a-f92a54fb3d5c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:52.573671 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72332cdb-13fc-4fec-988a-f92a54fb3d5c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:52.573947 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72332cdb-13fc-4fec-988a-f92a54fb3d5c None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:52.575410 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72332cdb-13fc-4fec-988a-f92a54fb3d5c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:52.575726 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72332cdb-13fc-4fec-988a-f92a54fb3d5c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:52.577797 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72332cdb-13fc-4fec-988a-f92a54fb3d5c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:52.595111 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 363/710] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 4 09:06:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:52.601306 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72332cdb-13fc-4fec-988a-f92a54fb3d5c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:52.626287 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eced5f-8dce-4b21-9cc6-6e9d57bd5f2f None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:52.663679 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1561298-d741-4738-8e1d-fea40abdf455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:52.665072 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1561298-d741-4738-8e1d-fea40abdf455 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:52.665356 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1561298-d741-4738-8e1d-fea40abdf455 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:52.665631 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1561298-d741-4738-8e1d-fea40abdf455 None admin] PATH_INFO: `/v3/services/network` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:52.666287 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1561298-d741-4738-8e1d-fea40abdf455 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:52.667674 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1561298-d741-4738-8e1d-fea40abdf455 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:52.682089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-f1561298-d741-4738-8e1d-fea40abdf455 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 04 09:06:52.682089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:52.682089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:06:52.682089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:06:52.682089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:06:52.682089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:06:52.682089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:06:52.682089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:06:52.682089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 04 09:06:52.682089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:52.682089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:06:52.682089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:52.682089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:52.682089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:52.682089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:52.682089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:52.682089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:52.683173 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:52.683173 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:52.683173 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:52.683173 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:52.683173 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:52.683173 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:52.683173 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:06:52.683173 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:06:52.683173 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:06:52.683173 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:06:52.683173 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:52.683173 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:52.683173 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:52.683173 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:52.683173 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:52.683173 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:52.684025 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:52.684025 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:52.684025 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:52.684025 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:52.684025 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:52.684025 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:52.684025 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:52.684025 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:52.684025 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:52.684025 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:52.684025 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:06:52.684025 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:06:52.684025 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 04 09:06:52.684025 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:52.688239 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 348/711] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 4 09:06:52 2022] GET /identity/v3/services/network => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:52.689444 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1561298-d741-4738-8e1d-fea40abdf455 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:52.694042 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:52.737702 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72332cdb-13fc-4fec-988a-f92a54fb3d5c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:52.758577 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:52.784497 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2519ebf-1d49-4110-be54-b838681e0b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:52.785200 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2519ebf-1d49-4110-be54-b838681e0b13 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:52.785486 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2519ebf-1d49-4110-be54-b838681e0b13 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:52.785763 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2519ebf-1d49-4110-be54-b838681e0b13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:52.786608 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2519ebf-1d49-4110-be54-b838681e0b13 None admin] RBAC: Adding query filter params (role.id=f36e053b442540f08041ac7878760f65, scope.project.id=4076e316243946e980ce47386b18b28f, user.id=15039da209ec445d93d49e2bd6d854eb) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:52.786922 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2519ebf-1d49-4110-be54-b838681e0b13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:52.788369 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2519ebf-1d49-4110-be54-b838681e0b13 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:52.803213 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 364/712] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 4 09:06:52 2022] GET /identity/v3/role_assignments?user.id=15039da209ec445d93d49e2bd6d854eb&scope.project.id=4076e316243946e980ce47386b18b28f&role.id=f36e053b442540f08041ac7878760f65 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:06:52.822879 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381cdb8-5585-4919-9e23-03cf750bfd46 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:52.823166 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381cdb8-5585-4919-9e23-03cf750bfd46 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:52.823344 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381cdb8-5585-4919-9e23-03cf750bfd46 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:52.825353 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 365/713] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:52.834614 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976bd7dd-b41e-47fb-938b-1602780b0aa8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:52.834951 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976bd7dd-b41e-47fb-938b-1602780b0aa8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:52.835182 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976bd7dd-b41e-47fb-938b-1602780b0aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:52.869608 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1561298-d741-4738-8e1d-fea40abdf455 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:52.878115 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-976bd7dd-b41e-47fb-938b-1602780b0aa8 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:52.879351 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-976bd7dd-b41e-47fb-938b-1602780b0aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:52.908936 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a1d23b-7427-4ff9-b52b-edc230da6eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:52.909649 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a1d23b-7427-4ff9-b52b-edc230da6eed None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:52.909938 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a1d23b-7427-4ff9-b52b-edc230da6eed None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:52.910216 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a1d23b-7427-4ff9-b52b-edc230da6eed None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:52.911262 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a1d23b-7427-4ff9-b52b-edc230da6eed None admin] RBAC: Adding query filter params (name=network) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:52.911573 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a1d23b-7427-4ff9-b52b-edc230da6eed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:52.913618 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a1d23b-7427-4ff9-b52b-edc230da6eed None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:52.920633 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 349/714] 10.222.0.71 () {56 vars in 1112 bytes} [Mon Jul 4 09:06:52 2022] GET /identity/v3/services?name=network => generated 124 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:52.930626 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a1d23b-7427-4ff9-b52b-edc230da6eed None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:52.992584 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 366/715] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:06:53.019921 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976bd7dd-b41e-47fb-938b-1602780b0aa8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:53.053168 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:53.094681 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a1d23b-7427-4ff9-b52b-edc230da6eed None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:53.131820 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd13940-aa72-43ee-94b9-21730e4caf1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:53.133517 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd13940-aa72-43ee-94b9-21730e4caf1b None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:53.134032 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd13940-aa72-43ee-94b9-21730e4caf1b None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:53.134579 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd13940-aa72-43ee-94b9-21730e4caf1b None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:53.135491 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd13940-aa72-43ee-94b9-21730e4caf1b None admin] RBAC: Adding query filter params (type=network) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:53.136017 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd13940-aa72-43ee-94b9-21730e4caf1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:53.137803 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd13940-aa72-43ee-94b9-21730e4caf1b None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:53.144298 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 350/716] 10.222.0.71 () {56 vars in 1112 bytes} [Mon Jul 4 09:06:52 2022] GET /identity/v3/services?type=network => generated 124 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:53.171425 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976bd7dd-b41e-47fb-938b-1602780b0aa8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:53.185972 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:53.209658 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0129b83b-6949-48dd-9eac-b6bc0fc41a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:53.210497 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0129b83b-6949-48dd-9eac-b6bc0fc41a58 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:53.211307 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0129b83b-6949-48dd-9eac-b6bc0fc41a58 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:53.211586 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0129b83b-6949-48dd-9eac-b6bc0fc41a58 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:53.212510 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0129b83b-6949-48dd-9eac-b6bc0fc41a58 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:53.213936 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0129b83b-6949-48dd-9eac-b6bc0fc41a58 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:53.229293 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-0129b83b-6949-48dd-9eac-b6bc0fc41a58 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 04 09:06:53.229293 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:53.229293 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:06:53.229293 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:06:53.229293 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:06:53.229293 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:06:53.229293 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:06:53.229293 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:06:53.229293 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 04 09:06:53.229293 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:53.229293 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:06:53.229293 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:53.229293 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:53.229293 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:53.229293 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:53.229293 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:53.230322 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:53.230322 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:53.230322 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:53.230322 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:53.230322 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:53.230322 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:53.230322 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:53.230322 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:06:53.230322 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:06:53.230322 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:06:53.230322 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:06:53.230322 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:53.230322 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:53.230322 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:53.230322 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:53.230322 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:53.231212 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:53.231212 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:53.231212 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:53.231212 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:53.231212 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:53.231212 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:53.231212 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:53.231212 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:53.231212 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:53.231212 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:53.231212 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:53.231212 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:06:53.231212 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:06:53.231212 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 04 09:06:53.231212 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:53.231212 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 367/717] 10.222.0.71 () {56 vars in 1129 bytes} [Mon Jul 4 09:06:53 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:06:53.238645 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd13940-aa72-43ee-94b9-21730e4caf1b None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:53.370896 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd13940-aa72-43ee-94b9-21730e4caf1b None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:53.402749 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:53.408251 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff89ef-73a4-4b2f-bce3-371c39716351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:53.409068 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff89ef-73a4-4b2f-bce3-371c39716351 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:53.409363 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff89ef-73a4-4b2f-bce3-371c39716351 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:53.413349 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff89ef-73a4-4b2f-bce3-371c39716351 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:53.413349 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ff89ef-73a4-4b2f-bce3-371c39716351 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:53.413349 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ff89ef-73a4-4b2f-bce3-371c39716351 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:53.413349 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ff89ef-73a4-4b2f-bce3-371c39716351 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:53.418601 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 351/718] 10.222.0.71 () {56 vars in 1124 bytes} [Mon Jul 4 09:06:53 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:53.424087 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0129b83b-6949-48dd-9eac-b6bc0fc41a58 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:53.548598 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0129b83b-6949-48dd-9eac-b6bc0fc41a58 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:53.597228 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3aad37-5832-4386-ad22-79d5b7e099d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:53.597904 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3aad37-5832-4386-ad22-79d5b7e099d4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:53.598190 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3aad37-5832-4386-ad22-79d5b7e099d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:53.598472 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3aad37-5832-4386-ad22-79d5b7e099d4 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:53.599187 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3aad37-5832-4386-ad22-79d5b7e099d4 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:53.599497 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3aad37-5832-4386-ad22-79d5b7e099d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:53.600939 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3aad37-5832-4386-ad22-79d5b7e099d4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:53.611771 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 368/719] 10.222.0.71 () {56 vars in 1124 bytes} [Mon Jul 4 09:06:53 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:54.796397 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666cbdad-7b77-4c81-b57b-d0b79306598a None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:54.796972 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666cbdad-7b77-4c81-b57b-d0b79306598a None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:54.796972 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666cbdad-7b77-4c81-b57b-d0b79306598a None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:54.798592 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 352/720] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:54.806718 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5682b1dd-b89a-49c3-8367-59535a9c2c02 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:54.807161 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5682b1dd-b89a-49c3-8367-59535a9c2c02 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:54.807345 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5682b1dd-b89a-49c3-8367-59535a9c2c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:54.838252 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5682b1dd-b89a-49c3-8367-59535a9c2c02 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:54.839457 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5682b1dd-b89a-49c3-8367-59535a9c2c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:54.908515 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:54.963797 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 369/721] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:06:55.006493 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666cbdad-7b77-4c81-b57b-d0b79306598a None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:55.080717 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0088ba00-0d81-411b-a176-e68b96142b03 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:55.080717 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0088ba00-0d81-411b-a176-e68b96142b03 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:55.080717 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0088ba00-0d81-411b-a176-e68b96142b03 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:55.081468 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 370/722] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:55.092070 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29139412-a3f6-4239-b070-6b0c64bda3c9 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:55.092673 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29139412-a3f6-4239-b070-6b0c64bda3c9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:55.093188 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29139412-a3f6-4239-b070-6b0c64bda3c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:55.120381 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-29139412-a3f6-4239-b070-6b0c64bda3c9 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:55.121997 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-29139412-a3f6-4239-b070-6b0c64bda3c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:55.157315 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666cbdad-7b77-4c81-b57b-d0b79306598a None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:55.199757 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d37b2-0600-4cdf-abcc-88f9a3018ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:55.200621 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d37b2-0600-4cdf-abcc-88f9a3018ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:55.200911 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d37b2-0600-4cdf-abcc-88f9a3018ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:55.201328 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d37b2-0600-4cdf-abcc-88f9a3018ac7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:55.213208 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50d37b2-0600-4cdf-abcc-88f9a3018ac7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:06:55.221237 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-f50d37b2-0600-4cdf-abcc-88f9a3018ac7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:06:55.221237 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:55.221237 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:55.221237 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:55.221237 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:55.221237 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:55.221237 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:55.221237 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:55.221237 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:55.221237 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:55.221237 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:55.221237 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:55.221237 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:06:55.221237 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:06:55.221237 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:06:55.221237 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:06:55.222196 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:06:55.222196 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:06:55.222196 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:06:55.222196 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:06:55.222196 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:55.222196 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:55.222196 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:55.222196 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:55.222196 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:55.222196 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:55.222196 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:55.222196 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:55.222196 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:55.222196 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:55.222196 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:55.222196 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:55.223039 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:55.223039 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:55.223039 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:55.223039 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:55.223039 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:06:55.223039 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:06:55.223039 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:06:55.223039 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:06:55.223039 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:06:55.223039 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:06:55.223039 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:06:55.223039 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:55.223039 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 353/723] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 4 09:06:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:55.230393 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d37b2-0600-4cdf-abcc-88f9a3018ac7 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:55.255379 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 371/724] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:06:55.280352 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29139412-a3f6-4239-b070-6b0c64bda3c9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:55.324525 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:55.391514 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d37b2-0600-4cdf-abcc-88f9a3018ac7 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:55.423713 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:55.427505 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29139412-a3f6-4239-b070-6b0c64bda3c9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:55.436969 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c6fdd9-087c-4f7b-9c23-6ede6ef4236d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:55.437672 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c6fdd9-087c-4f7b-9c23-6ede6ef4236d None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:55.437955 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c6fdd9-087c-4f7b-9c23-6ede6ef4236d None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:55.438231 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c6fdd9-087c-4f7b-9c23-6ede6ef4236d None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:55.439334 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c6fdd9-087c-4f7b-9c23-6ede6ef4236d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:55.439651 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c6fdd9-087c-4f7b-9c23-6ede6ef4236d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:55.441432 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c6fdd9-087c-4f7b-9c23-6ede6ef4236d None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:55.453799 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 354/725] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:06:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:06:55.456203 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b02f5-5b8c-4800-a89e-97950929db74 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:55.456671 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b02f5-5b8c-4800-a89e-97950929db74 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:55.457127 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b02f5-5b8c-4800-a89e-97950929db74 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:55.464148 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 355/726] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:55.464148 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037b02f5-5b8c-4800-a89e-97950929db74 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:55.470068 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bff566-c701-427c-af22-8414d5158aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:55.470921 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bff566-c701-427c-af22-8414d5158aa4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:55.471250 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bff566-c701-427c-af22-8414d5158aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:55.471534 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bff566-c701-427c-af22-8414d5158aa4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:55.483690 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bff566-c701-427c-af22-8414d5158aa4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:06:55.492988 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-33bff566-c701-427c-af22-8414d5158aa4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:06:55.492988 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:55.492988 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:55.492988 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:55.492988 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:55.492988 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:55.492988 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:55.492988 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:55.492988 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:55.492988 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:55.492988 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:55.492988 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:55.492988 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:06:55.492988 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:06:55.492988 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:06:55.492988 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:06:55.493997 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:06:55.493997 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:06:55.493997 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:06:55.493997 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:06:55.493997 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:55.493997 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:55.493997 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:55.493997 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:55.493997 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:55.493997 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:55.493997 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:55.493997 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:55.493997 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:55.493997 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:55.493997 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:55.493997 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:55.494859 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:55.494859 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:55.494859 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:55.494859 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:55.494859 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:06:55.494859 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:06:55.494859 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:06:55.494859 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:06:55.494859 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:06:55.494859 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:06:55.494859 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:06:55.494859 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:55.494859 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 372/727] 10.222.0.71 () {56 vars in 1102 bytes} [Mon Jul 4 09:06:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:06:55.500589 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e9915-a065-4935-8c21-97ac949690f7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:55.500922 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e9915-a065-4935-8c21-97ac949690f7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:55.501181 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e9915-a065-4935-8c21-97ac949690f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:55.520536 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ba5e9915-a065-4935-8c21-97ac949690f7 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:55.521782 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ba5e9915-a065-4935-8c21-97ac949690f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:55.600138 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 373/728] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:06:55.600138 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e9915-a065-4935-8c21-97ac949690f7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:55.614985 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037b02f5-5b8c-4800-a89e-97950929db74 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:55.645012 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:55.667918 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d88fa75-d383-4b37-9e20-4008e2473541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:55.668729 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88fa75-d383-4b37-9e20-4008e2473541 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:55.669019 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88fa75-d383-4b37-9e20-4008e2473541 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:55.669326 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88fa75-d383-4b37-9e20-4008e2473541 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:55.680067 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d88fa75-d383-4b37-9e20-4008e2473541 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:55.681522 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d88fa75-d383-4b37-9e20-4008e2473541 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:55.699516 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-1d88fa75-d383-4b37-9e20-4008e2473541 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:55.699516 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:55.699516 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:55.699516 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:55.699516 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:55.699516 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:55.699516 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:55.699516 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:55.699516 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:55.699516 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:55.699516 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:55.699516 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:55.699516 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:06:55.699516 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:06:55.699516 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:06:55.700453 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:06:55.700453 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:55.700453 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:55.700453 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:55.700453 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:55.700453 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:55.700453 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:55.700453 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:55.700453 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:55.700453 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:55.700453 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:55.700453 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:55.700453 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:55.700453 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:55.700453 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:55.700453 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:55.700453 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:55.701405 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:06:55.701405 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:06:55.701405 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:55.701405 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:55.701405 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 356/729] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:06:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:55.701699 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d88fa75-d383-4b37-9e20-4008e2473541 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:55.726108 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:55.738653 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e9915-a065-4935-8c21-97ac949690f7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:55.776124 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc25ee2-31fb-40ab-94e4-6ff9dd96301c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:55.776801 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc25ee2-31fb-40ab-94e4-6ff9dd96301c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:55.777109 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc25ee2-31fb-40ab-94e4-6ff9dd96301c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:55.777391 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc25ee2-31fb-40ab-94e4-6ff9dd96301c None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:55.778143 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc25ee2-31fb-40ab-94e4-6ff9dd96301c None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:55.778450 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc25ee2-31fb-40ab-94e4-6ff9dd96301c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:55.779824 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc25ee2-31fb-40ab-94e4-6ff9dd96301c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:55.793317 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 374/730] 10.222.0.71 () {56 vars in 1103 bytes} [Mon Jul 4 09:06:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:55.794811 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2764d6a6-82d9-4028-b9b2-52bb708609dd None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:55.795101 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2764d6a6-82d9-4028-b9b2-52bb708609dd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:55.795382 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2764d6a6-82d9-4028-b9b2-52bb708609dd None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:55.797200 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 375/731] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:55.797524 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2764d6a6-82d9-4028-b9b2-52bb708609dd None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:55.807106 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d88fa75-d383-4b37-9e20-4008e2473541 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:55.843833 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ce5ee9-af98-43eb-a5e4-238927f2472d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:55.844635 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce5ee9-af98-43eb-a5e4-238927f2472d None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:55.844916 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce5ee9-af98-43eb-a5e4-238927f2472d None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:55.845233 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce5ee9-af98-43eb-a5e4-238927f2472d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:55.856181 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ce5ee9-af98-43eb-a5e4-238927f2472d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:06:55.863707 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-d7ce5ee9-af98-43eb-a5e4-238927f2472d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:06:55.863707 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:55.863707 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:55.863707 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:55.863707 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:55.863707 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:55.863707 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:55.863707 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:55.863707 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:55.863707 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:55.863707 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:55.863707 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:55.863707 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:06:55.863707 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:06:55.863707 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:06:55.863707 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:06:55.864660 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:06:55.864660 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:06:55.864660 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:06:55.864660 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:06:55.864660 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:55.864660 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:55.864660 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:55.864660 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:55.864660 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:55.864660 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:55.864660 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:55.864660 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:55.864660 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:55.864660 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:55.864660 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:55.864660 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:55.865882 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:55.865882 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:55.865882 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:55.865882 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:55.865882 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:06:55.865882 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:06:55.865882 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:06:55.865882 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:06:55.865882 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:06:55.865882 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:06:55.865882 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:06:55.865882 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:55.865882 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 357/732] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 4 09:06:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:55.865882 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ce5ee9-af98-43eb-a5e4-238927f2472d None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:55.899375 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:55.922262 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2764d6a6-82d9-4028-b9b2-52bb708609dd None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:55.963729 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc6f043-1cc0-46e9-b552-dc7d6b239bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:55.965178 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc6f043-1cc0-46e9-b552-dc7d6b239bb5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:55.965689 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc6f043-1cc0-46e9-b552-dc7d6b239bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:55.966294 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc6f043-1cc0-46e9-b552-dc7d6b239bb5 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:55.967081 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc6f043-1cc0-46e9-b552-dc7d6b239bb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:55.967609 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc6f043-1cc0-46e9-b552-dc7d6b239bb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:55.969242 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc6f043-1cc0-46e9-b552-dc7d6b239bb5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:55.971499 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ce5ee9-af98-43eb-a5e4-238927f2472d None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:55.986774 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 376/733] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:06:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:55.986774 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6627b9f3-3ebe-45ec-948b-e5d094f35b6f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:55.986774 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6627b9f3-3ebe-45ec-948b-e5d094f35b6f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:55.986774 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6627b9f3-3ebe-45ec-948b-e5d094f35b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:56.001570 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6627b9f3-3ebe-45ec-948b-e5d094f35b6f None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:56.002754 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6627b9f3-3ebe-45ec-948b-e5d094f35b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:56.011672 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:56.012411 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe483b4-4c10-4768-a66f-b448342c553e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:56.013385 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe483b4-4c10-4768-a66f-b448342c553e None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:56.013783 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe483b4-4c10-4768-a66f-b448342c553e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:56.014013 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe483b4-4c10-4768-a66f-b448342c553e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:56.026113 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe483b4-4c10-4768-a66f-b448342c553e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:56.027595 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe483b4-4c10-4768-a66f-b448342c553e None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:56.046132 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-bbe483b4-4c10-4768-a66f-b448342c553e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:56.046132 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:56.046132 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:56.046132 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:56.046132 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:56.046132 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:56.046132 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:56.046132 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:56.046132 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:56.046132 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:56.046132 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:56.046132 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:56.046132 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:06:56.046132 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:06:56.046132 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:06:56.047091 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:06:56.047091 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:56.047091 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:56.047091 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:56.047091 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:56.047091 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:56.047091 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:56.047091 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:56.047091 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:56.047091 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:56.047091 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:56.047091 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:56.047091 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:56.047091 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:56.047091 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:56.047091 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:56.047091 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:56.048005 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:06:56.048005 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:06:56.048005 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:56.048005 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:56.048005 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 358/734] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:06:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:56.048737 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe483b4-4c10-4768-a66f-b448342c553e None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:56.089264 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 377/735] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:06:56.089264 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6627b9f3-3ebe-45ec-948b-e5d094f35b6f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:56.142766 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:56.162068 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe483b4-4c10-4768-a66f-b448342c553e None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:56.197413 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0750d142-b844-489c-bd2d-9cd6bccbfb06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:56.198093 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0750d142-b844-489c-bd2d-9cd6bccbfb06 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:56.198380 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0750d142-b844-489c-bd2d-9cd6bccbfb06 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:56.198667 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0750d142-b844-489c-bd2d-9cd6bccbfb06 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:56.199417 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0750d142-b844-489c-bd2d-9cd6bccbfb06 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:56.199737 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0750d142-b844-489c-bd2d-9cd6bccbfb06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:56.201127 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0750d142-b844-489c-bd2d-9cd6bccbfb06 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:56.207918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6627b9f3-3ebe-45ec-948b-e5d094f35b6f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:56.209306 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 359/736] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:06:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:56.210100 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0750d142-b844-489c-bd2d-9cd6bccbfb06 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:56.251707 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48afea63-458b-483d-974b-95b2fa835896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:56.252505 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48afea63-458b-483d-974b-95b2fa835896 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:56.252786 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48afea63-458b-483d-974b-95b2fa835896 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:56.253084 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48afea63-458b-483d-974b-95b2fa835896 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:56.260926 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48afea63-458b-483d-974b-95b2fa835896 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:56.262401 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48afea63-458b-483d-974b-95b2fa835896 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:56.276651 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-48afea63-458b-483d-974b-95b2fa835896 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 04 09:06:56.276651 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:56.276651 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:56.276651 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:56.276651 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:56.276651 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:56.276651 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:56.276651 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:56.276651 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:56.276651 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:56.276651 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:56.276651 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:56.276651 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:06:56.276651 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:06:56.276651 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:06:56.276651 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:06:56.277789 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:56.277789 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:56.277789 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:06:56.277789 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:06:56.277789 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:06:56.277789 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:06:56.277789 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:56.277789 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:56.277789 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:56.277789 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:56.277789 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:56.277789 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:56.277789 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:56.277789 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:56.277789 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:56.277789 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:56.277789 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:56.277789 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:56.279204 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:56.279204 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:56.279204 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:06:56.279204 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:06:56.279204 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:06:56.279204 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:06:56.279204 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:06:56.279204 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:06:56.279204 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 04 09:06:56.279204 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:56.279204 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 378/737] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 4 09:06:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:56.279204 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48afea63-458b-483d-974b-95b2fa835896 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:56.281384 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:56.315033 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0750d142-b844-489c-bd2d-9cd6bccbfb06 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:56.348638 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a03e3-95d1-48cd-9403-e87c7e16616c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:56.349335 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a03e3-95d1-48cd-9403-e87c7e16616c None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:56.349623 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a03e3-95d1-48cd-9403-e87c7e16616c None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:56.349903 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a03e3-95d1-48cd-9403-e87c7e16616c None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:56.350629 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72a03e3-95d1-48cd-9403-e87c7e16616c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:56.350939 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72a03e3-95d1-48cd-9403-e87c7e16616c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:56.352348 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72a03e3-95d1-48cd-9403-e87c7e16616c None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:56.363825 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 360/738] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:06:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:56.364610 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a03e3-95d1-48cd-9403-e87c7e16616c None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:56.370017 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:56.377483 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48afea63-458b-483d-974b-95b2fa835896 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:56.413065 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98045cce-255e-426c-b1fa-10f06fdf58b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:56.414191 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98045cce-255e-426c-b1fa-10f06fdf58b4 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:56.414511 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98045cce-255e-426c-b1fa-10f06fdf58b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:56.414798 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98045cce-255e-426c-b1fa-10f06fdf58b4 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:56.415525 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98045cce-255e-426c-b1fa-10f06fdf58b4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:56.416900 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98045cce-255e-426c-b1fa-10f06fdf58b4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:56.433450 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 379/739] 10.222.0.71 () {60 vars in 1141 bytes} [Mon Jul 4 09:06:56 2022] POST /identity/v3/services => generated 240 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:06:56.434739 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca442d-2b81-487c-9db8-6688353ad597 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:56.435053 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca442d-2b81-487c-9db8-6688353ad597 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:56.435456 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca442d-2b81-487c-9db8-6688353ad597 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:56.436712 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 380/740] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:56.437491 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eca442d-2b81-487c-9db8-6688353ad597 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:56.475019 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a03e3-95d1-48cd-9403-e87c7e16616c None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:56.516706 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:56.530472 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c53f32d-9786-42d8-94a5-73548f842b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:56.531303 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53f32d-9786-42d8-94a5-73548f842b8a None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:56.531595 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53f32d-9786-42d8-94a5-73548f842b8a None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:56.531879 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53f32d-9786-42d8-94a5-73548f842b8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:56.543569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c53f32d-9786-42d8-94a5-73548f842b8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:56.545180 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c53f32d-9786-42d8-94a5-73548f842b8a None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:56.565610 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-9c53f32d-9786-42d8-94a5-73548f842b8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:56.565610 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:56.565610 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:56.565610 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:56.565610 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:56.565610 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:56.565610 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:56.565610 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:56.565610 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:56.565610 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:56.565610 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:56.565610 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:56.565610 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:06:56.565610 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:06:56.565610 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:06:56.566587 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:06:56.566587 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:56.566587 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:56.566587 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:56.566587 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:56.566587 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:56.566587 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:56.566587 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:56.566587 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:56.566587 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:56.566587 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:56.566587 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:56.566587 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:56.566587 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:56.566587 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:56.566587 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:56.566587 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:56.567513 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:06:56.567513 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:06:56.567513 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:56.567513 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:56.567513 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 361/741] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:06:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:56.567797 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c53f32d-9786-42d8-94a5-73548f842b8a None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:56.573808 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eca442d-2b81-487c-9db8-6688353ad597 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:56.615873 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577891d8-22d1-41a4-b804-18de44837b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:56.617498 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577891d8-22d1-41a4-b804-18de44837b1e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:56.617918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577891d8-22d1-41a4-b804-18de44837b1e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:56.618386 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577891d8-22d1-41a4-b804-18de44837b1e None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:56.619076 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577891d8-22d1-41a4-b804-18de44837b1e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:56.619395 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577891d8-22d1-41a4-b804-18de44837b1e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:56.620864 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577891d8-22d1-41a4-b804-18de44837b1e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:56.631138 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:56.640967 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 381/742] 10.222.0.71 () {56 vars in 1135 bytes} [Mon Jul 4 09:06:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:56.645867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114d833-e511-4577-b930-ca3f26994dd5 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:56.646500 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114d833-e511-4577-b930-ca3f26994dd5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:56.647282 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114d833-e511-4577-b930-ca3f26994dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:56.664276 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b114d833-e511-4577-b930-ca3f26994dd5 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:56.665585 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b114d833-e511-4577-b930-ca3f26994dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:56.684266 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c53f32d-9786-42d8-94a5-73548f842b8a None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:56.722246 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7585841-fcb8-4d21-acec-3a92683c47f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:56.723472 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7585841-fcb8-4d21-acec-3a92683c47f0 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:56.724063 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7585841-fcb8-4d21-acec-3a92683c47f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:56.724491 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7585841-fcb8-4d21-acec-3a92683c47f0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:56.733316 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7585841-fcb8-4d21-acec-3a92683c47f0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:56.734718 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 382/743] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:56 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:06:56.735133 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7585841-fcb8-4d21-acec-3a92683c47f0 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:56.735635 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114d833-e511-4577-b930-ca3f26994dd5 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:56.761569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-f7585841-fcb8-4d21-acec-3a92683c47f0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 09:06:56.761569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:56.761569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:56.761569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:56.761569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:56.761569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:56.761569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:56.761569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:56.761569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:56.761569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:56.761569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:56.761569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:56.761569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:06:56.761569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:06:56.761569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:06:56.761569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:06:56.762803 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:56.762803 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:56.762803 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:06:56.762803 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:06:56.762803 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:06:56.762803 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:06:56.762803 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:56.762803 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:56.762803 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:56.762803 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:56.762803 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:56.762803 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:56.762803 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:56.762803 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:56.762803 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:56.762803 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:56.762803 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:56.762803 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:56.763985 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:56.763985 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:56.763985 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:06:56.763985 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:06:56.763985 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:06:56.763985 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:06:56.763985 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:06:56.763985 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:06:56.763985 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 09:06:56.763985 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:56.765988 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 362/744] 10.222.0.71 () {56 vars in 1117 bytes} [Mon Jul 4 09:06:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:56.767014 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7585841-fcb8-4d21-acec-3a92683c47f0 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:56.797866 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:56.852154 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114d833-e511-4577-b930-ca3f26994dd5 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:56.884189 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7585841-fcb8-4d21-acec-3a92683c47f0 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:56.888762 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b652708-e12b-470d-99f5-182492985494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:56.889468 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b652708-e12b-470d-99f5-182492985494 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:56.889755 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b652708-e12b-470d-99f5-182492985494 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:56.890033 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b652708-e12b-470d-99f5-182492985494 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:56.890757 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b652708-e12b-470d-99f5-182492985494 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:56.891064 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b652708-e12b-470d-99f5-182492985494 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:56.892447 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b652708-e12b-470d-99f5-182492985494 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:56.898181 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:56.908771 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 383/745] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:06:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:56.908771 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b652708-e12b-470d-99f5-182492985494 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:56.925422 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ccf5a8-b607-4a43-8c85-7f768b349519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:56.926049 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ccf5a8-b607-4a43-8c85-7f768b349519 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:56.926333 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ccf5a8-b607-4a43-8c85-7f768b349519 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:56.926613 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ccf5a8-b607-4a43-8c85-7f768b349519 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:57.003017 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ccf5a8-b607-4a43-8c85-7f768b349519 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:57.004730 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ccf5a8-b607-4a43-8c85-7f768b349519 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:57.016807 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b652708-e12b-470d-99f5-182492985494 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:57.030579 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:57.063565 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2a9717-a6c1-4e94-b996-eeec2a10129d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:57.064397 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2a9717-a6c1-4e94-b996-eeec2a10129d None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:57.064781 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2a9717-a6c1-4e94-b996-eeec2a10129d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:57.065201 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2a9717-a6c1-4e94-b996-eeec2a10129d None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:57.066029 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2a9717-a6c1-4e94-b996-eeec2a10129d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:57.068073 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2a9717-a6c1-4e94-b996-eeec2a10129d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:57.094200 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 384/746] 10.222.0.71 () {60 vars in 1141 bytes} [Mon Jul 4 09:06:56 2022] POST /identity/v3/services => generated 251 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:06:57.095229 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2a9717-a6c1-4e94-b996-eeec2a10129d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:57.146390 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 363/747] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:06:56 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:06:57.147919 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ccf5a8-b607-4a43-8c85-7f768b349519 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:57.173528 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:57.261324 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2a9717-a6c1-4e94-b996-eeec2a10129d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:57.317897 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cd0b85-e16e-479c-9b52-3f5c828b2346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:57.318701 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd0b85-e16e-479c-9b52-3f5c828b2346 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:57.318995 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd0b85-e16e-479c-9b52-3f5c828b2346 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:57.319278 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd0b85-e16e-479c-9b52-3f5c828b2346 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:57.322142 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cd0b85-e16e-479c-9b52-3f5c828b2346 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:57.322475 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cd0b85-e16e-479c-9b52-3f5c828b2346 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:57.322896 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ccf5a8-b607-4a43-8c85-7f768b349519 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:57.324117 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cd0b85-e16e-479c-9b52-3f5c828b2346 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:57.348734 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 385/748] 10.222.0.71 () {56 vars in 1149 bytes} [Mon Jul 4 09:06:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:57.349929 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cd0b85-e16e-479c-9b52-3f5c828b2346 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:57.369250 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201b7dc1-a6c8-4b42-9ce6-9c33b3af7c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:57.370116 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b7dc1-a6c8-4b42-9ce6-9c33b3af7c53 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:57.370453 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b7dc1-a6c8-4b42-9ce6-9c33b3af7c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:57.370746 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b7dc1-a6c8-4b42-9ce6-9c33b3af7c53 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:57.378830 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201b7dc1-a6c8-4b42-9ce6-9c33b3af7c53 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:57.380293 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201b7dc1-a6c8-4b42-9ce6-9c33b3af7c53 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:57.389665 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:57.397236 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-201b7dc1-a6c8-4b42-9ce6-9c33b3af7c53 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 04 09:06:57.397236 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:57.397236 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:57.397236 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:57.397236 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:57.397236 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:57.397236 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:57.397236 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:57.397236 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:57.397236 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:57.397236 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:57.397236 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:57.397236 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:06:57.397236 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:06:57.397236 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:06:57.397236 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:06:57.399374 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:57.399374 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:57.399374 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:06:57.399374 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:06:57.399374 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:06:57.399374 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:06:57.399374 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:57.399374 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:57.399374 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:57.399374 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:57.399374 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:57.399374 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:57.399374 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:57.399374 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:57.399374 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:57.399374 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:57.399374 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:57.399374 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:57.400586 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:57.400586 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:57.400586 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:06:57.400586 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:06:57.400586 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:06:57.400586 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:06:57.400586 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:06:57.400586 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:06:57.400586 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 04 09:06:57.400586 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:57.400586 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 364/749] 10.222.0.71 () {56 vars in 1093 bytes} [Mon Jul 4 09:06:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:57.400586 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201b7dc1-a6c8-4b42-9ce6-9c33b3af7c53 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:57.482686 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cd0b85-e16e-479c-9b52-3f5c828b2346 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:57.526116 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76c600a-9049-495c-9151-be6f83aad584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:57.527955 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76c600a-9049-495c-9151-be6f83aad584 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:57.528738 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76c600a-9049-495c-9151-be6f83aad584 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:57.529264 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76c600a-9049-495c-9151-be6f83aad584 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:57.531614 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:57.547049 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76c600a-9049-495c-9151-be6f83aad584 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:57.549680 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76c600a-9049-495c-9151-be6f83aad584 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:57.574575 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201b7dc1-a6c8-4b42-9ce6-9c33b3af7c53 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:57.575028 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-c76c600a-9049-495c-9151-be6f83aad584 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:57.575028 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:57.575028 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:57.575028 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:57.575028 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:57.575028 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:57.575028 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:57.575028 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:57.575028 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:57.575028 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:57.575028 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:57.575028 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:57.575028 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:06:57.575028 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:06:57.575028 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:06:57.576444 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:06:57.576444 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:57.576444 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:57.576444 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:57.576444 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:57.576444 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:57.576444 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:57.576444 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:57.576444 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:57.576444 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:57.576444 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:57.576444 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:57.576444 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:57.576444 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:57.576444 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:57.576444 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:57.576444 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:57.578457 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:06:57.578457 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:06:57.578457 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:57.578457 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:57.578457 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 386/750] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:06:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:57.578457 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76c600a-9049-495c-9151-be6f83aad584 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:57.633629 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4002d42b-3bbc-43f2-acdf-856cf3382e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:57.635092 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4002d42b-3bbc-43f2-acdf-856cf3382e10 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:57.635945 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4002d42b-3bbc-43f2-acdf-856cf3382e10 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:57.636658 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4002d42b-3bbc-43f2-acdf-856cf3382e10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:57.682534 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:57.714035 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76c600a-9049-495c-9151-be6f83aad584 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:57.718484 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4002d42b-3bbc-43f2-acdf-856cf3382e10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:57.720235 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4002d42b-3bbc-43f2-acdf-856cf3382e10 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:57.763466 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562787ce-bb32-46e4-967f-27a7bdbc8a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:57.764396 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562787ce-bb32-46e4-967f-27a7bdbc8a32 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:57.764689 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562787ce-bb32-46e4-967f-27a7bdbc8a32 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:57.764970 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562787ce-bb32-46e4-967f-27a7bdbc8a32 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:57.766755 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562787ce-bb32-46e4-967f-27a7bdbc8a32 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:57.767075 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562787ce-bb32-46e4-967f-27a7bdbc8a32 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:57.768534 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562787ce-bb32-46e4-967f-27a7bdbc8a32 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:57.792426 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 387/751] 10.222.0.71 () {56 vars in 1133 bytes} [Mon Jul 4 09:06:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:57.793492 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562787ce-bb32-46e4-967f-27a7bdbc8a32 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:57.805369 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:57.843229 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 365/752] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:06:57 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:06:57.844385 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4002d42b-3bbc-43f2-acdf-856cf3382e10 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:57.931513 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562787ce-bb32-46e4-967f-27a7bdbc8a32 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:57.961287 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4002d42b-3bbc-43f2-acdf-856cf3382e10 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:57.980081 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f65edf-ff88-4f69-a3fc-65925d5e426f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:57.981029 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f65edf-ff88-4f69-a3fc-65925d5e426f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:57.981358 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f65edf-ff88-4f69-a3fc-65925d5e426f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:57.981637 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f65edf-ff88-4f69-a3fc-65925d5e426f None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:57.982371 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f65edf-ff88-4f69-a3fc-65925d5e426f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:57.982680 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f65edf-ff88-4f69-a3fc-65925d5e426f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:57.984753 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f65edf-ff88-4f69-a3fc-65925d5e426f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:57.986530 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:58.003968 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced2085e-46fa-4db2-a85b-789cb82aae3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:58.005131 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2085e-46fa-4db2-a85b-789cb82aae3b None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:58.005619 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2085e-46fa-4db2-a85b-789cb82aae3b None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:58.006116 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2085e-46fa-4db2-a85b-789cb82aae3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:58.006539 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 388/753] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:06:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:58.007477 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f65edf-ff88-4f69-a3fc-65925d5e426f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:58.018377 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced2085e-46fa-4db2-a85b-789cb82aae3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:58.020249 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced2085e-46fa-4db2-a85b-789cb82aae3b None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:58.040731 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-ced2085e-46fa-4db2-a85b-789cb82aae3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:58.040731 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:58.040731 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:58.040731 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:58.040731 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:58.040731 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:58.040731 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:58.040731 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:58.040731 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:58.040731 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:58.040731 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:58.040731 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:58.040731 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:06:58.040731 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:06:58.040731 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:06:58.041723 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:06:58.041723 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:58.041723 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:58.041723 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:58.041723 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:58.041723 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:58.041723 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:58.041723 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:58.041723 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:58.041723 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:58.041723 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:58.041723 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:58.041723 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:58.041723 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:58.041723 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:58.041723 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:58.041723 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:58.042679 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:06:58.042679 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:06:58.042679 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:58.042679 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:58.045485 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 366/754] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:06:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:58.047327 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced2085e-46fa-4db2-a85b-789cb82aae3b None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:58.082341 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:58.135621 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f65edf-ff88-4f69-a3fc-65925d5e426f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:58.167724 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced2085e-46fa-4db2-a85b-789cb82aae3b None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:58.178965 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f17037e-d87a-40e0-89a3-a23f12c52846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:58.179815 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f17037e-d87a-40e0-89a3-a23f12c52846 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:58.180100 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f17037e-d87a-40e0-89a3-a23f12c52846 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:58.180381 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f17037e-d87a-40e0-89a3-a23f12c52846 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:58.198194 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f17037e-d87a-40e0-89a3-a23f12c52846 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:58.199981 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f17037e-d87a-40e0-89a3-a23f12c52846 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:58.213088 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71d0216-e32d-4c5a-87cb-64d56c79de4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:58.214200 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71d0216-e32d-4c5a-87cb-64d56c79de4e None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:58.214680 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71d0216-e32d-4c5a-87cb-64d56c79de4e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:58.215152 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71d0216-e32d-4c5a-87cb-64d56c79de4e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:58.222234 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71d0216-e32d-4c5a-87cb-64d56c79de4e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:58.222727 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-5f17037e-d87a-40e0-89a3-a23f12c52846 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:58.222727 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:58.222727 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:58.222727 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:58.222727 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:58.222727 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:58.222727 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:58.222727 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:58.222727 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:58.222727 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:58.222727 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:58.222727 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:58.222727 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:06:58.222727 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:06:58.222727 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:06:58.223607 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:06:58.223607 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:58.223607 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:58.223607 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:58.223607 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:58.223607 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:58.223607 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:58.223607 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:58.223607 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:58.223607 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:58.223607 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:58.223607 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:58.223607 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:58.223607 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:58.223607 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:58.223607 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:58.223607 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:58.224578 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:06:58.224578 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:06:58.224578 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:06:58.224578 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:58.224578 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 389/755] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:06:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:58.224859 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f17037e-d87a-40e0-89a3-a23f12c52846 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:58.226840 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71d0216-e32d-4c5a-87cb-64d56c79de4e None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:58.250621 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-b71d0216-e32d-4c5a-87cb-64d56c79de4e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:06:58.250621 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:58.250621 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:58.250621 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:58.250621 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:58.250621 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:58.250621 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:58.250621 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:58.250621 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:58.250621 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:58.250621 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:58.250621 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:58.250621 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:06:58.250621 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:06:58.250621 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:06:58.251569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:06:58.251569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:58.251569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:58.251569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:58.251569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:58.251569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:58.251569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:58.251569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:58.251569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:58.251569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:58.251569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:58.251569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:58.251569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:58.251569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:58.251569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:58.251569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:58.251569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:58.252493 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:06:58.252493 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:06:58.252493 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:06:58.252493 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:06:58.252493 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:06:58.252493 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:06:58.252493 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:06:58.252493 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:58.254881 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 367/756] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 4 09:06:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:58.255817 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71d0216-e32d-4c5a-87cb-64d56c79de4e None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:58.284359 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:58.344716 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f17037e-d87a-40e0-89a3-a23f12c52846 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:58.372494 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:58.383689 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2ab586-66d4-4377-b3e8-6d8d339d7ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:58.384665 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ab586-66d4-4377-b3e8-6d8d339d7ea2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:58.385236 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ab586-66d4-4377-b3e8-6d8d339d7ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:58.386073 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ab586-66d4-4377-b3e8-6d8d339d7ea2 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:58.387197 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2ab586-66d4-4377-b3e8-6d8d339d7ea2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:58.387612 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2ab586-66d4-4377-b3e8-6d8d339d7ea2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:58.389374 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2ab586-66d4-4377-b3e8-6d8d339d7ea2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:58.394906 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71d0216-e32d-4c5a-87cb-64d56c79de4e None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:58.404780 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 390/757] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:06:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:58.406368 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2ab586-66d4-4377-b3e8-6d8d339d7ea2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:58.435407 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5286c3-d1c3-45b2-8d87-1fec8dc3a721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:58.436552 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5286c3-d1c3-45b2-8d87-1fec8dc3a721 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:58.436987 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5286c3-d1c3-45b2-8d87-1fec8dc3a721 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:58.437467 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5286c3-d1c3-45b2-8d87-1fec8dc3a721 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:58.438502 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5286c3-d1c3-45b2-8d87-1fec8dc3a721 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:58.438955 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5286c3-d1c3-45b2-8d87-1fec8dc3a721 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:58.440670 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5286c3-d1c3-45b2-8d87-1fec8dc3a721 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:58.455066 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 368/758] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:06:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:58.456087 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5286c3-d1c3-45b2-8d87-1fec8dc3a721 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:58.532993 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2ab586-66d4-4377-b3e8-6d8d339d7ea2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:58.541437 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:58.572155 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5523d6ba-d5bb-470b-931c-55bca668dbc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:58.572969 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5523d6ba-d5bb-470b-931c-55bca668dbc1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:58.573379 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5523d6ba-d5bb-470b-931c-55bca668dbc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:58.573760 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5523d6ba-d5bb-470b-931c-55bca668dbc1 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:58.574296 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5286c3-d1c3-45b2-8d87-1fec8dc3a721 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:58.574979 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5523d6ba-d5bb-470b-931c-55bca668dbc1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:58.575297 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5523d6ba-d5bb-470b-931c-55bca668dbc1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:58.576749 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5523d6ba-d5bb-470b-931c-55bca668dbc1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:58.588976 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 391/759] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 4 09:06:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:58.590086 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5523d6ba-d5bb-470b-931c-55bca668dbc1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:58.615542 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5601ec9-0007-46c5-8ad9-27a2a3cfd7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:58.616227 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5601ec9-0007-46c5-8ad9-27a2a3cfd7b1 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:58.616508 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5601ec9-0007-46c5-8ad9-27a2a3cfd7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:58.617567 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5601ec9-0007-46c5-8ad9-27a2a3cfd7b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:58.624913 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5601ec9-0007-46c5-8ad9-27a2a3cfd7b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:58.626459 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5601ec9-0007-46c5-8ad9-27a2a3cfd7b1 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:58.640905 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-a5601ec9-0007-46c5-8ad9-27a2a3cfd7b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:06:58.640905 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:58.640905 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:58.640905 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:58.640905 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:58.640905 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:58.640905 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:58.640905 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:58.640905 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:58.640905 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:58.640905 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:58.640905 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:58.640905 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:06:58.640905 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:06:58.640905 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:06:58.641930 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:06:58.641930 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:58.641930 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:58.641930 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:58.641930 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:58.641930 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:58.641930 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:58.641930 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:58.641930 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:58.641930 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:58.641930 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:58.641930 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:58.641930 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:58.641930 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:58.641930 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:58.641930 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:58.641930 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:58.642828 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:06:58.642828 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:06:58.642828 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:06:58.642828 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:06:58.642828 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:06:58.642828 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:06:58.642828 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:06:58.642828 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:06:58.642828 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 369/760] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 4 09:06:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:58.643324 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2421094-54da-442d-9cae-90314123db77 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:58.643852 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2421094-54da-442d-9cae-90314123db77 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:58.643852 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2421094-54da-442d-9cae-90314123db77 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:58.649446 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 370/761] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:58.649446 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2421094-54da-442d-9cae-90314123db77 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:58.653297 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:58.703604 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5523d6ba-d5bb-470b-931c-55bca668dbc1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:58.739791 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0c4b73-738e-41b4-80d5-4ffe1b4e4ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:58.740582 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c4b73-738e-41b4-80d5-4ffe1b4e4ed4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:58.740873 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c4b73-738e-41b4-80d5-4ffe1b4e4ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:58.741189 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c4b73-738e-41b4-80d5-4ffe1b4e4ed4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:58.748756 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0c4b73-738e-41b4-80d5-4ffe1b4e4ed4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:58.750280 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0c4b73-738e-41b4-80d5-4ffe1b4e4ed4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:58.758652 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2421094-54da-442d-9cae-90314123db77 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:58.764701 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-2a0c4b73-738e-41b4-80d5-4ffe1b4e4ed4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:06:58.764701 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:58.764701 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:58.764701 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:58.764701 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:58.764701 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:58.764701 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:58.764701 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:58.764701 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:58.764701 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:58.764701 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:58.764701 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:58.764701 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:06:58.764701 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:06:58.764701 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:06:58.765603 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:06:58.765603 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:58.765603 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:58.765603 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:58.765603 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:58.765603 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:58.765603 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:58.765603 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:58.765603 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:58.765603 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:58.765603 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:58.765603 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:58.765603 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:58.765603 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:58.765603 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:58.765603 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:58.765603 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:58.766542 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:06:58.766542 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:06:58.766542 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:06:58.766542 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:06:58.766542 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:06:58.766542 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:06:58.766542 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:06:58.766542 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:58.766542 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 392/762] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 4 09:06:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:58.767993 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0c4b73-738e-41b4-80d5-4ffe1b4e4ed4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:58.795565 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:58.801448 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee19bdf-de0b-41de-ad62-7bbd4f5a1233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:58.802117 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee19bdf-de0b-41de-ad62-7bbd4f5a1233 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:58.802406 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee19bdf-de0b-41de-ad62-7bbd4f5a1233 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:58.802838 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee19bdf-de0b-41de-ad62-7bbd4f5a1233 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:58.803836 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee19bdf-de0b-41de-ad62-7bbd4f5a1233 None admin] RBAC: Adding query filter params (role.id=f36e053b442540f08041ac7878760f65, scope.project.id=4076e316243946e980ce47386b18b28f, user.id=88b41b55769a4c449716c3590ed58225) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:58.804160 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee19bdf-de0b-41de-ad62-7bbd4f5a1233 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:58.806340 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee19bdf-de0b-41de-ad62-7bbd4f5a1233 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:58.821454 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 371/763] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 4 09:06:58 2022] GET /identity/v3/role_assignments?user.id=88b41b55769a4c449716c3590ed58225&scope.project.id=4076e316243946e980ce47386b18b28f&role.id=f36e053b442540f08041ac7878760f65 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:58.821454 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ebd9a-9201-49fb-a967-092100092b1c None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:58.821454 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ebd9a-9201-49fb-a967-092100092b1c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:58.821454 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ebd9a-9201-49fb-a967-092100092b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:58.840284 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-f61ebd9a-9201-49fb-a967-092100092b1c None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:58.841666 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f61ebd9a-9201-49fb-a967-092100092b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:58.894518 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0c4b73-738e-41b4-80d5-4ffe1b4e4ed4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:58.912817 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:58.921212 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 372/764] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:06:58.922330 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61ebd9a-9201-49fb-a967-092100092b1c None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:58.942088 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5265cfdf-302e-4951-b1de-c2e7eff6869f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:58.943358 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5265cfdf-302e-4951-b1de-c2e7eff6869f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:58.943797 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5265cfdf-302e-4951-b1de-c2e7eff6869f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:58.944227 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5265cfdf-302e-4951-b1de-c2e7eff6869f None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:58.945759 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5265cfdf-302e-4951-b1de-c2e7eff6869f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:58.946293 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5265cfdf-302e-4951-b1de-c2e7eff6869f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:58.948652 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5265cfdf-302e-4951-b1de-c2e7eff6869f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:58.971813 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 393/765] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 4 09:06:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:58.971813 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5265cfdf-302e-4951-b1de-c2e7eff6869f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:59.042577 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61ebd9a-9201-49fb-a967-092100092b1c None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:59.071002 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:59.086386 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4c4309-d739-4dbf-8752-0bef4ced94ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:59.087557 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c4309-d739-4dbf-8752-0bef4ced94ab None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:59.088013 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c4309-d739-4dbf-8752-0bef4ced94ab None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:59.088435 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c4309-d739-4dbf-8752-0bef4ced94ab None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:59.088559 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5265cfdf-302e-4951-b1de-c2e7eff6869f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:59.089612 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4c4309-d739-4dbf-8752-0bef4ced94ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:59.090317 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4c4309-d739-4dbf-8752-0bef4ced94ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:59.092886 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4c4309-d739-4dbf-8752-0bef4ced94ab None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:59.108748 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 373/766] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 4 09:06:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:59.109907 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4c4309-d739-4dbf-8752-0bef4ced94ab None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:59.129325 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951f1064-6547-4e79-aa5b-ad0a9153d9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:59.130308 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f1064-6547-4e79-aa5b-ad0a9153d9e7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:59.130712 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f1064-6547-4e79-aa5b-ad0a9153d9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:59.131084 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f1064-6547-4e79-aa5b-ad0a9153d9e7 None admin] PATH_INFO: `/v3/services/network` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:59.131848 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951f1064-6547-4e79-aa5b-ad0a9153d9e7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:59.133921 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951f1064-6547-4e79-aa5b-ad0a9153d9e7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:59.146330 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-951f1064-6547-4e79-aa5b-ad0a9153d9e7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 04 09:06:59.146330 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:59.146330 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:06:59.146330 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:06:59.146330 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:06:59.146330 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:06:59.146330 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:06:59.146330 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:06:59.146330 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 04 09:06:59.146330 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:59.146330 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:06:59.146330 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:59.146330 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:59.146330 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:59.146330 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:59.146330 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:59.146330 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:59.148087 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:59.148087 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:59.148087 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:59.148087 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:59.148087 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:59.148087 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:59.148087 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:06:59.148087 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:06:59.148087 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:06:59.148087 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:06:59.148087 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:59.148087 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:59.148087 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:59.148087 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:59.148087 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:59.148087 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:59.149277 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:59.149277 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:59.149277 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:59.149277 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:59.149277 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:59.149277 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:59.149277 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:59.149277 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:59.149277 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:59.149277 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:59.149277 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:06:59.149277 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:06:59.149277 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 04 09:06:59.149277 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:59.150751 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 394/767] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 4 09:06:58 2022] GET /identity/v3/services/network => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:06:59.151643 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951f1064-6547-4e79-aa5b-ad0a9153d9e7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:59.190948 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:59.240537 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4c4309-d739-4dbf-8752-0bef4ced94ab None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:59.269912 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951f1064-6547-4e79-aa5b-ad0a9153d9e7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:59.277014 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e45da90-902b-4d41-85ef-3b71713674ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:59.278043 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45da90-902b-4d41-85ef-3b71713674ff None admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:59.278423 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45da90-902b-4d41-85ef-3b71713674ff None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:59.278796 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45da90-902b-4d41-85ef-3b71713674ff None admin] PATH_INFO: `/v3/projects/4076e316243946e980ce47386b18b28f/users/15039da209ec445d93d49e2bd6d854eb/roles/f36e053b442540f08041ac7878760f65` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:59.280856 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e45da90-902b-4d41-85ef-3b71713674ff None admin] RBAC: Authorizing `identity:create_grant(project_id=4076e316243946e980ce47386b18b28f, user_id=15039da209ec445d93d49e2bd6d854eb, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:59.282449 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e45da90-902b-4d41-85ef-3b71713674ff None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:59.297271 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 374/768] 10.222.0.71 () {58 vars in 1439 bytes} [Mon Jul 4 09:06:59 2022] PUT /identity/v3/projects/4076e316243946e980ce47386b18b28f/users/15039da209ec445d93d49e2bd6d854eb/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:06:59.298464 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e45da90-902b-4d41-85ef-3b71713674ff None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:59.308131 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628ca16f-5b4d-459b-aa8b-5feac25b0042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:59.308865 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ca16f-5b4d-459b-aa8b-5feac25b0042 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:59.309229 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ca16f-5b4d-459b-aa8b-5feac25b0042 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:59.309518 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ca16f-5b4d-459b-aa8b-5feac25b0042 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:59.310689 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628ca16f-5b4d-459b-aa8b-5feac25b0042 None admin] RBAC: Adding query filter params (role.id=5752a9bdfe4b482ab37836eba7bd536b, scope.project.id=4076e316243946e980ce47386b18b28f, user.id=78668b939c5847b1b73e9c28b5d8db7f) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:59.311039 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628ca16f-5b4d-459b-aa8b-5feac25b0042 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:59.312584 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628ca16f-5b4d-459b-aa8b-5feac25b0042 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:59.319478 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 395/769] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 4 09:06:59 2022] GET /identity/v3/role_assignments?user.id=78668b939c5847b1b73e9c28b5d8db7f&scope.project.id=4076e316243946e980ce47386b18b28f&role.id=5752a9bdfe4b482ab37836eba7bd536b => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:59.405785 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826db27-5b1f-41bf-af87-d2ada011be99 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:59.405989 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826db27-5b1f-41bf-af87-d2ada011be99 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:59.406252 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826db27-5b1f-41bf-af87-d2ada011be99 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:59.407982 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 396/770] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:06:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:06:59.413557 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d74213-631e-4a67-b6f5-652f4b3363bd None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:59.414379 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d74213-631e-4a67-b6f5-652f4b3363bd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:59.415325 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d74213-631e-4a67-b6f5-652f4b3363bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:59.427392 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e45da90-902b-4d41-85ef-3b71713674ff None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:59.434370 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e3d74213-631e-4a67-b6f5-652f4b3363bd None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:59.435855 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e3d74213-631e-4a67-b6f5-652f4b3363bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:59.447813 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:59.453486 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:59.472614 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb77f00-6300-4f03-97b7-e4b512bf0bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:59.473526 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb77f00-6300-4f03-97b7-e4b512bf0bcd None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:59.473910 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb77f00-6300-4f03-97b7-e4b512bf0bcd None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:59.474320 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb77f00-6300-4f03-97b7-e4b512bf0bcd None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:59.475177 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb77f00-6300-4f03-97b7-e4b512bf0bcd None admin] RBAC: Adding query filter params (name=network) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:59.475753 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb77f00-6300-4f03-97b7-e4b512bf0bcd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:59.477605 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb77f00-6300-4f03-97b7-e4b512bf0bcd None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:59.483793 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 375/771] 10.222.0.71 () {56 vars in 1112 bytes} [Mon Jul 4 09:06:59 2022] GET /identity/v3/services?name=network => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:06:59.490262 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb77f00-6300-4f03-97b7-e4b512bf0bcd None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:59.579897 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 397/772] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:06:59.604747 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d74213-631e-4a67-b6f5-652f4b3363bd None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:59.651743 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb77f00-6300-4f03-97b7-e4b512bf0bcd None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:59.709798 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f940bff4-0242-44a4-ad10-f68802c247eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:59.711518 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940bff4-0242-44a4-ad10-f68802c247eb None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:59.712015 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940bff4-0242-44a4-ad10-f68802c247eb None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:59.712701 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940bff4-0242-44a4-ad10-f68802c247eb None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:59.713593 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f940bff4-0242-44a4-ad10-f68802c247eb None admin] RBAC: Adding query filter params (type=network) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:59.714061 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f940bff4-0242-44a4-ad10-f68802c247eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:59.715588 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f940bff4-0242-44a4-ad10-f68802c247eb None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:59.726807 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 376/773] 10.222.0.71 () {56 vars in 1112 bytes} [Mon Jul 4 09:06:59 2022] GET /identity/v3/services?type=network => generated 350 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:06:59.735294 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f940bff4-0242-44a4-ad10-f68802c247eb None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:59.791248 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d74213-631e-4a67-b6f5-652f4b3363bd None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:59.795663 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:59.856111 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f940bff4-0242-44a4-ad10-f68802c247eb None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:59.856452 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a177ae6-1ebd-4de9-96a3-e962f0f64647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:59.857323 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a177ae6-1ebd-4de9-96a3-e962f0f64647 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:59.857610 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a177ae6-1ebd-4de9-96a3-e962f0f64647 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:59.857889 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a177ae6-1ebd-4de9-96a3-e962f0f64647 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:59.858630 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a177ae6-1ebd-4de9-96a3-e962f0f64647 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:59.860278 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a177ae6-1ebd-4de9-96a3-e962f0f64647 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:59.881845 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-9a177ae6-1ebd-4de9-96a3-e962f0f64647 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 04 09:06:59.881845 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:59.881845 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:06:59.881845 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:06:59.881845 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:06:59.881845 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:06:59.881845 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:06:59.881845 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:06:59.881845 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 04 09:06:59.881845 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:59.881845 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:06:59.881845 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:59.881845 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:06:59.881845 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:06:59.881845 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:06:59.881845 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:06:59.883551 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:06:59.883551 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:06:59.883551 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:06:59.883551 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:06:59.883551 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:06:59.883551 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:06:59.883551 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:06:59.883551 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:06:59.883551 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:06:59.883551 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:06:59.883551 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:06:59.883551 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:06:59.883551 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:06:59.883551 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:06:59.883551 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:06:59.883551 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:06:59.884430 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:06:59.884430 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:06:59.884430 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:06:59.884430 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:06:59.884430 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:06:59.884430 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:06:59.884430 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:06:59.884430 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:06:59.884430 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:06:59.884430 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:06:59.884430 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:06:59.884430 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:06:59.884430 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:06:59.884430 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 04 09:06:59.884430 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:06:59.888220 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 398/774] 10.222.0.71 () {56 vars in 1129 bytes} [Mon Jul 4 09:06:59 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:06:59.889369 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a177ae6-1ebd-4de9-96a3-e962f0f64647 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:59.904577 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:59.906291 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c98cb5-65e3-458b-b77b-7ea2fe2d1a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:59.907167 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c98cb5-65e3-458b-b77b-7ea2fe2d1a0e None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:59.907320 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c98cb5-65e3-458b-b77b-7ea2fe2d1a0e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:59.907613 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c98cb5-65e3-458b-b77b-7ea2fe2d1a0e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:59.908472 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c98cb5-65e3-458b-b77b-7ea2fe2d1a0e None admin] RBAC: Adding query filter params (interface=public, service_id=9f1d281c01c64818a0ec5da6ed4d4ccd, region_id=RegionOne) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:06:59.908798 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c98cb5-65e3-458b-b77b-7ea2fe2d1a0e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:59.910355 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c98cb5-65e3-458b-b77b-7ea2fe2d1a0e None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:59.926651 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 377/775] 10.222.0.71 () {56 vars in 1251 bytes} [Mon Jul 4 09:06:59 2022] GET /identity/v3/endpoints?service_id=9f1d281c01c64818a0ec5da6ed4d4ccd&interface=public®ion_id=RegionOne => generated 194 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:06:59.933546 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c98cb5-65e3-458b-b77b-7ea2fe2d1a0e None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:00.021550 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a177ae6-1ebd-4de9-96a3-e962f0f64647 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:00.091254 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738f4699-01cc-40ac-8345-479555e50c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:00.092275 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738f4699-01cc-40ac-8345-479555e50c9f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:00.092573 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738f4699-01cc-40ac-8345-479555e50c9f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:00.092861 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738f4699-01cc-40ac-8345-479555e50c9f None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:00.094170 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738f4699-01cc-40ac-8345-479555e50c9f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:00.094489 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738f4699-01cc-40ac-8345-479555e50c9f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:00.096509 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738f4699-01cc-40ac-8345-479555e50c9f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:00.108238 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 399/776] 10.222.0.71 () {56 vars in 1124 bytes} [Mon Jul 4 09:06:59 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:00.110531 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c98cb5-65e3-458b-b77b-7ea2fe2d1a0e None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:00.111931 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738f4699-01cc-40ac-8345-479555e50c9f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:00.133204 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:00.156346 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c31424-d51d-4381-8ede-894ad77e01b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:00.157329 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c31424-d51d-4381-8ede-894ad77e01b0 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:00.157760 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c31424-d51d-4381-8ede-894ad77e01b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:00.158188 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c31424-d51d-4381-8ede-894ad77e01b0 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:00.159027 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c31424-d51d-4381-8ede-894ad77e01b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:00.160770 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c31424-d51d-4381-8ede-894ad77e01b0 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:00.167266 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 378/777] 10.222.0.71 () {56 vars in 1087 bytes} [Mon Jul 4 09:06:59 2022] GET /identity/v3/services => generated 772 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:07:00.263786 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:00.312961 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738f4699-01cc-40ac-8345-479555e50c9f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:00.354047 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51a9213-ad2e-40d8-8326-244fba2194d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:00.354729 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51a9213-ad2e-40d8-8326-244fba2194d6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:00.355023 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51a9213-ad2e-40d8-8326-244fba2194d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:00.355287 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51a9213-ad2e-40d8-8326-244fba2194d6 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:00.356005 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51a9213-ad2e-40d8-8326-244fba2194d6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:00.356314 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51a9213-ad2e-40d8-8326-244fba2194d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:00.357719 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51a9213-ad2e-40d8-8326-244fba2194d6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:00.363137 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 400/778] 10.222.0.71 () {56 vars in 1124 bytes} [Mon Jul 4 09:07:00 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:00.369686 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c31424-d51d-4381-8ede-894ad77e01b0 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:00.496531 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c31424-d51d-4381-8ede-894ad77e01b0 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:00.542010 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e60afee-ab94-479d-9c58-5e90fae41227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:00.542729 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60afee-ab94-479d-9c58-5e90fae41227 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:00.543024 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60afee-ab94-479d-9c58-5e90fae41227 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:00.543303 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60afee-ab94-479d-9c58-5e90fae41227 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:00.544097 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e60afee-ab94-479d-9c58-5e90fae41227 None admin] RBAC: Adding query filter params (interface=public, service_id=bfec4005f9fe4920891dc565be35f5e8, region_id=RegionOne) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:00.544410 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e60afee-ab94-479d-9c58-5e90fae41227 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:00.545829 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e60afee-ab94-479d-9c58-5e90fae41227 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:00.551286 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 379/779] 10.222.0.71 () {56 vars in 1251 bytes} [Mon Jul 4 09:07:00 2022] GET /identity/v3/endpoints?service_id=bfec4005f9fe4920891dc565be35f5e8&interface=public®ion_id=RegionOne => generated 194 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:00.558165 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51a9213-ad2e-40d8-8326-244fba2194d6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:00.715045 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51a9213-ad2e-40d8-8326-244fba2194d6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:00.764518 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725966e2-a891-44ee-86d7-3b63eea791ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:00.765303 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725966e2-a891-44ee-86d7-3b63eea791ca None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:00.765595 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725966e2-a891-44ee-86d7-3b63eea791ca None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:00.765875 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725966e2-a891-44ee-86d7-3b63eea791ca None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:00.766556 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725966e2-a891-44ee-86d7-3b63eea791ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:00.767916 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725966e2-a891-44ee-86d7-3b63eea791ca None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:00.769829 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:00.777398 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 401/780] 10.222.0.71 () {56 vars in 1087 bytes} [Mon Jul 4 09:07:00 2022] GET /identity/v3/services => generated 772 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:01.095453 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c0c945-35ed-4e6d-b65f-b86d5957a977 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:01.095661 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c0c945-35ed-4e6d-b65f-b86d5957a977 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:01.096021 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c0c945-35ed-4e6d-b65f-b86d5957a977 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:01.097581 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 380/781] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:07:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:01.102771 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c7757b-c208-4b24-ad8f-9a9e37065925 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:01.103226 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c7757b-c208-4b24-ad8f-9a9e37065925 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:01.103649 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c7757b-c208-4b24-ad8f-9a9e37065925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:01.122802 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-11c7757b-c208-4b24-ad8f-9a9e37065925 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:01.123994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-11c7757b-c208-4b24-ad8f-9a9e37065925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:01.204941 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 402/782] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:07:01.246254 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c0c945-35ed-4e6d-b65f-b86d5957a977 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:01.303020 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:01.393927 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c0c945-35ed-4e6d-b65f-b86d5957a977 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:01.440673 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84614dcf-dd0a-4e6d-ab27-f1b689a5cf34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:01.441516 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84614dcf-dd0a-4e6d-ab27-f1b689a5cf34 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:01.441802 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84614dcf-dd0a-4e6d-ab27-f1b689a5cf34 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:01.442083 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84614dcf-dd0a-4e6d-ab27-f1b689a5cf34 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:01.458239 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84614dcf-dd0a-4e6d-ab27-f1b689a5cf34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:07:01.466995 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-84614dcf-dd0a-4e6d-ab27-f1b689a5cf34 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:07:01.466995 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:01.466995 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:01.466995 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:01.466995 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:01.466995 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:01.466995 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:01.466995 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:01.466995 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:01.466995 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:01.466995 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:01.466995 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:01.466995 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:07:01.466995 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:07:01.466995 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:07:01.466995 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:07:01.467923 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:07:01.467923 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:07:01.467923 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:07:01.467923 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:07:01.467923 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:01.467923 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:01.467923 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:01.467923 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:01.467923 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:01.467923 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:01.467923 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:01.467923 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:01.467923 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:01.467923 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:01.467923 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:01.467923 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:01.468773 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:01.468773 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:01.468773 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:01.468773 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:01.468773 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:07:01.468773 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:07:01.468773 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:07:01.468773 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:07:01.468773 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:07:01.468773 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:07:01.468773 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:07:01.468773 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:01.468773 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 381/783] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 4 09:07:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:01.478172 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c7757b-c208-4b24-ad8f-9a9e37065925 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:01.630959 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c7757b-c208-4b24-ad8f-9a9e37065925 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:01.670717 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:01.678716 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b5acd4-07e1-4ec0-bfaf-71ede1523222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:01.679455 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5acd4-07e1-4ec0-bfaf-71ede1523222 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:01.679761 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5acd4-07e1-4ec0-bfaf-71ede1523222 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:01.680121 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5acd4-07e1-4ec0-bfaf-71ede1523222 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:01.680776 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b5acd4-07e1-4ec0-bfaf-71ede1523222 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:01.685457 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b5acd4-07e1-4ec0-bfaf-71ede1523222 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:01.686866 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b5acd4-07e1-4ec0-bfaf-71ede1523222 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:01.696443 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 403/784] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:07:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:01.701822 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84614dcf-dd0a-4e6d-ab27-f1b689a5cf34 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:01.857757 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:01.865307 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84614dcf-dd0a-4e6d-ab27-f1b689a5cf34 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:01.912827 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5962f6-4476-4a5e-9e5c-ef656e766fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:01.914085 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5962f6-4476-4a5e-9e5c-ef656e766fed None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:01.914575 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5962f6-4476-4a5e-9e5c-ef656e766fed None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:01.915056 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5962f6-4476-4a5e-9e5c-ef656e766fed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:01.932184 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5962f6-4476-4a5e-9e5c-ef656e766fed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:01.933950 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5962f6-4476-4a5e-9e5c-ef656e766fed None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:01.951905 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-0c5962f6-4476-4a5e-9e5c-ef656e766fed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:07:01.951905 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:01.951905 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:01.951905 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:01.951905 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:01.951905 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:01.951905 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:01.951905 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:01.951905 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:01.951905 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:01.951905 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:01.951905 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:01.951905 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:07:01.951905 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:07:01.951905 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:07:01.952849 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:07:01.952849 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:01.952849 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:01.952849 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:01.952849 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:01.952849 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:01.952849 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:01.952849 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:01.952849 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:01.952849 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:01.952849 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:01.952849 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:01.952849 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:01.952849 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:01.952849 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:01.952849 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:01.952849 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:01.953793 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:07:01.953793 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:07:01.953793 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:07:01.953793 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:01.956833 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 382/785] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:07:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:01.963169 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b5acd4-07e1-4ec0-bfaf-71ede1523222 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:02.109119 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b5acd4-07e1-4ec0-bfaf-71ede1523222 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:02.117771 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b79dac-5a82-4061-a0ee-fd1a60ebfd0b None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:02.118071 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b79dac-5a82-4061-a0ee-fd1a60ebfd0b None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:02.118344 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b79dac-5a82-4061-a0ee-fd1a60ebfd0b None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:02.119856 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 383/786] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:07:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:02.126075 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785946bf-5313-4296-91b2-7b2321dcd2f0 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:02.126370 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785946bf-5313-4296-91b2-7b2321dcd2f0 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:02.126658 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785946bf-5313-4296-91b2-7b2321dcd2f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:02.148752 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b2f9c0-9c1f-4add-859d-160c10870c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:02.149823 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b2f9c0-9c1f-4add-859d-160c10870c63 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:02.150334 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b2f9c0-9c1f-4add-859d-160c10870c63 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:02.150823 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b2f9c0-9c1f-4add-859d-160c10870c63 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:02.151772 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b2f9c0-9c1f-4add-859d-160c10870c63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:02.152286 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b2f9c0-9c1f-4add-859d-160c10870c63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:02.153922 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b2f9c0-9c1f-4add-859d-160c10870c63 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:02.162085 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-785946bf-5313-4296-91b2-7b2321dcd2f0 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:02.163330 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-785946bf-5313-4296-91b2-7b2321dcd2f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:02.167686 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 404/787] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:07:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:02.169243 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ade30-7e81-41cd-a02f-b6afcd6f7d3e None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:02.169600 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ade30-7e81-41cd-a02f-b6afcd6f7d3e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:02.169975 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ade30-7e81-41cd-a02f-b6afcd6f7d3e None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:02.171646 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 405/788] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:07:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:02.174008 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ade30-7e81-41cd-a02f-b6afcd6f7d3e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:02.234915 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 384/789] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:07:02.236236 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163df36f-331f-4ae8-a22d-fb67c47fc21f None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:02.236532 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163df36f-331f-4ae8-a22d-fb67c47fc21f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:02.236812 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163df36f-331f-4ae8-a22d-fb67c47fc21f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:02.249563 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:02.254117 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-163df36f-331f-4ae8-a22d-fb67c47fc21f None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:02.255381 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-163df36f-331f-4ae8-a22d-fb67c47fc21f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:02.301000 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ade30-7e81-41cd-a02f-b6afcd6f7d3e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:02.339242 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145eb6bc-288a-437e-9b51-69f883d17669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:02.340044 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145eb6bc-288a-437e-9b51-69f883d17669 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:02.340330 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145eb6bc-288a-437e-9b51-69f883d17669 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:02.340610 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145eb6bc-288a-437e-9b51-69f883d17669 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:02.345209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 385/790] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:07:02.346121 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163df36f-331f-4ae8-a22d-fb67c47fc21f None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:02.356814 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145eb6bc-288a-437e-9b51-69f883d17669 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:02.362610 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145eb6bc-288a-437e-9b51-69f883d17669 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:02.368723 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:02.384252 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-145eb6bc-288a-437e-9b51-69f883d17669 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 04 09:07:02.384252 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:02.384252 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:02.384252 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:02.384252 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:02.384252 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:02.384252 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:02.384252 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:02.384252 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:02.384252 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:02.384252 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:02.384252 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:02.384252 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:07:02.384252 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:07:02.384252 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:07:02.384252 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:07:02.386067 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:02.386067 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:02.386067 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:07:02.386067 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:07:02.386067 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:07:02.386067 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:07:02.386067 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:02.386067 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:02.386067 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:02.386067 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:02.386067 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:02.386067 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:02.386067 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:02.386067 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:02.386067 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:02.386067 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:02.386067 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:02.386067 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:02.387023 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:02.387023 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:02.387023 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:07:02.387023 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:07:02.387023 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:07:02.387023 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:07:02.387023 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:07:02.387023 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:07:02.387023 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 04 09:07:02.387023 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:07:02.387023 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 406/791] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 4 09:07:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:02.387023 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145eb6bc-288a-437e-9b51-69f883d17669 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:02.496104 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163df36f-331f-4ae8-a22d-fb67c47fc21f None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:02.504039 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145eb6bc-288a-437e-9b51-69f883d17669 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:02.518470 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:02.536090 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20181d1-5f91-4ed7-8158-aef55cf87493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:02.536918 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20181d1-5f91-4ed7-8158-aef55cf87493 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:02.537261 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20181d1-5f91-4ed7-8158-aef55cf87493 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:02.537546 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20181d1-5f91-4ed7-8158-aef55cf87493 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:02.542628 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd9f4ae-7fb1-4acc-a7c8-4a0be9a9f121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:02.543670 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd9f4ae-7fb1-4acc-a7c8-4a0be9a9f121 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:02.544181 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd9f4ae-7fb1-4acc-a7c8-4a0be9a9f121 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:02.544841 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd9f4ae-7fb1-4acc-a7c8-4a0be9a9f121 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:02.550024 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20181d1-5f91-4ed7-8158-aef55cf87493 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:07:02.556555 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd9f4ae-7fb1-4acc-a7c8-4a0be9a9f121 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:07:02.558776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-a20181d1-5f91-4ed7-8158-aef55cf87493 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:07:02.558776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:02.558776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:02.558776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:02.558776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:02.558776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:02.558776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:02.558776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:02.558776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:02.558776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:02.558776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:02.558776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:02.558776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:07:02.558776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:07:02.558776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:07:02.558776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:07:02.559720 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:07:02.559720 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:07:02.559720 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:07:02.559720 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:07:02.559720 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:02.559720 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:02.559720 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:02.559720 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:02.559720 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:02.559720 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:02.559720 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:02.559720 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:02.559720 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:02.559720 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:02.559720 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:02.559720 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:02.561706 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:02.561706 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:02.561706 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:02.561706 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:02.561706 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:07:02.561706 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:07:02.561706 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:07:02.561706 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:07:02.561706 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:07:02.561706 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:07:02.561706 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:07:02.561706 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:02.561706 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 386/792] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 4 09:07:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:02.561706 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20181d1-5f91-4ed7-8158-aef55cf87493 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:02.569276 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-dfd9f4ae-7fb1-4acc-a7c8-4a0be9a9f121 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:07:02.569276 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:02.569276 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:02.569276 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:02.569276 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:02.569276 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:02.569276 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:02.569276 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:02.569276 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:02.569276 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:02.569276 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:02.569276 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:02.569276 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:07:02.569276 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:07:02.569276 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:07:02.569276 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:07:02.570281 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:07:02.570281 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:07:02.570281 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:07:02.570281 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:07:02.570281 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:02.570281 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:02.570281 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:02.570281 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:02.570281 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:02.570281 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:02.570281 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:02.570281 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:02.570281 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:02.570281 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:02.570281 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:02.570281 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:02.571139 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:02.571139 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:02.571139 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:02.571139 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:02.571139 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:07:02.571139 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:07:02.571139 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:07:02.571139 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:07:02.571139 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:07:02.571139 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:07:02.571139 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:07:02.571139 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:07:02.575100 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 407/793] 10.222.0.71 () {56 vars in 1102 bytes} [Mon Jul 4 09:07:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:02.576260 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd9f4ae-7fb1-4acc-a7c8-4a0be9a9f121 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:02.639180 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:02.675804 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20181d1-5f91-4ed7-8158-aef55cf87493 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:02.683614 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd9f4ae-7fb1-4acc-a7c8-4a0be9a9f121 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:02.713433 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8442ec12-7d17-4485-8aff-6253a91d4706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:02.714098 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8442ec12-7d17-4485-8aff-6253a91d4706 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:02.714384 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8442ec12-7d17-4485-8aff-6253a91d4706 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:02.714670 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8442ec12-7d17-4485-8aff-6253a91d4706 None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:02.715509 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8442ec12-7d17-4485-8aff-6253a91d4706 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:02.715829 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8442ec12-7d17-4485-8aff-6253a91d4706 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:02.717299 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8442ec12-7d17-4485-8aff-6253a91d4706 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:02.718897 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85435fd-6caf-4c63-a931-23cedd6273a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:02.719560 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85435fd-6caf-4c63-a931-23cedd6273a1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:02.720420 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85435fd-6caf-4c63-a931-23cedd6273a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:02.720420 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85435fd-6caf-4c63-a931-23cedd6273a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:02.720844 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85435fd-6caf-4c63-a931-23cedd6273a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:02.721182 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85435fd-6caf-4c63-a931-23cedd6273a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:02.722520 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85435fd-6caf-4c63-a931-23cedd6273a1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:02.735531 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 408/794] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:07:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:02.735531 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85435fd-6caf-4c63-a931-23cedd6273a1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:02.739943 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 387/795] 10.222.0.71 () {56 vars in 1135 bytes} [Mon Jul 4 09:07:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:02.745105 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2ed25a-6483-4a88-b3b3-cc9306a4d613 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:02.745105 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2ed25a-6483-4a88-b3b3-cc9306a4d613 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:02.745105 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2ed25a-6483-4a88-b3b3-cc9306a4d613 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:02.745105 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 388/796] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:07:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:02.745105 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2ed25a-6483-4a88-b3b3-cc9306a4d613 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:02.777835 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:02.850956 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85435fd-6caf-4c63-a931-23cedd6273a1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:02.852769 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2ed25a-6483-4a88-b3b3-cc9306a4d613 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:02.881918 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:02.888663 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf33ef7-2b47-451b-b62b-4405e997d7c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:02.889359 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf33ef7-2b47-451b-b62b-4405e997d7c0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:02.889645 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf33ef7-2b47-451b-b62b-4405e997d7c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:02.889948 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf33ef7-2b47-451b-b62b-4405e997d7c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:02.890654 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf33ef7-2b47-451b-b62b-4405e997d7c0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:02.890961 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf33ef7-2b47-451b-b62b-4405e997d7c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:02.891076 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52aa3db-e398-454f-a476-82dd88c5cf24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:02.891831 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52aa3db-e398-454f-a476-82dd88c5cf24 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:02.892117 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52aa3db-e398-454f-a476-82dd88c5cf24 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:02.892321 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf33ef7-2b47-451b-b62b-4405e997d7c0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:02.892419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52aa3db-e398-454f-a476-82dd88c5cf24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:02.905095 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 409/797] 10.222.0.71 () {56 vars in 1103 bytes} [Mon Jul 4 09:07:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:02.905095 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf33ef7-2b47-451b-b62b-4405e997d7c0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:02.908052 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52aa3db-e398-454f-a476-82dd88c5cf24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:02.909574 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52aa3db-e398-454f-a476-82dd88c5cf24 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:02.928665 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-b52aa3db-e398-454f-a476-82dd88c5cf24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:07:02.928665 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:02.928665 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:02.928665 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:02.928665 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:02.928665 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:02.928665 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:02.928665 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:02.928665 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:02.928665 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:02.928665 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:02.928665 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:02.928665 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:07:02.928665 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:07:02.928665 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:07:02.930586 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:07:02.930586 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:02.930586 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:02.930586 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:02.930586 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:02.930586 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:02.930586 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:02.930586 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:02.930586 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:02.930586 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:02.930586 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:02.930586 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:02.930586 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:02.930586 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:02.930586 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:02.930586 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:02.930586 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:02.931531 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:07:02.931531 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:07:02.931531 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:07:02.931531 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:02.931531 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 389/798] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:07:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:02.932995 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d890b922-9c9f-437d-8ee0-96517ebfe477 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:02.932995 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d890b922-9c9f-437d-8ee0-96517ebfe477 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:02.932995 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d890b922-9c9f-437d-8ee0-96517ebfe477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:02.949856 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-d890b922-9c9f-437d-8ee0-96517ebfe477 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:02.951089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d890b922-9c9f-437d-8ee0-96517ebfe477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:03.010808 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:03.017797 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf33ef7-2b47-451b-b62b-4405e997d7c0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:03.025542 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 390/799] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:07:03.026462 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d890b922-9c9f-437d-8ee0-96517ebfe477 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:03.066157 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2b4a56-add4-4576-829b-39f195740e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:03.066877 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2b4a56-add4-4576-829b-39f195740e7e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:03.067162 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2b4a56-add4-4576-829b-39f195740e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:03.067478 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2b4a56-add4-4576-829b-39f195740e7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:03.080219 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2b4a56-add4-4576-829b-39f195740e7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:03.081702 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2b4a56-add4-4576-829b-39f195740e7e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:03.100212 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-7e2b4a56-add4-4576-829b-39f195740e7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:07:03.100212 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:03.100212 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:03.100212 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:03.100212 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:03.100212 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:03.100212 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:03.100212 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:03.100212 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:03.100212 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:03.100212 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:03.100212 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:03.100212 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:07:03.100212 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:07:03.100212 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:07:03.101258 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:07:03.101258 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:03.101258 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:03.101258 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:03.101258 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:03.101258 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:03.101258 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:03.101258 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:03.101258 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:03.101258 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:03.101258 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:03.101258 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:03.101258 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:03.101258 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:03.101258 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:03.101258 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:03.101258 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:03.102164 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:07:03.102164 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:07:03.102164 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:07:03.102164 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:07:03.102164 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 410/800] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:07:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:03.102530 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2b4a56-add4-4576-829b-39f195740e7e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:03.143286 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d890b922-9c9f-437d-8ee0-96517ebfe477 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:03.155852 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:03.182766 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9343186c-7b86-42a3-b8ec-bb0e0614d034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:03.183573 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9343186c-7b86-42a3-b8ec-bb0e0614d034 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:03.183859 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9343186c-7b86-42a3-b8ec-bb0e0614d034 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:03.184138 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9343186c-7b86-42a3-b8ec-bb0e0614d034 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:03.195753 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9343186c-7b86-42a3-b8ec-bb0e0614d034 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:03.197261 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9343186c-7b86-42a3-b8ec-bb0e0614d034 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:03.211068 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2b4a56-add4-4576-829b-39f195740e7e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:03.215159 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-9343186c-7b86-42a3-b8ec-bb0e0614d034 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:07:03.215159 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:03.215159 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:03.215159 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:03.215159 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:03.215159 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:03.215159 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:03.215159 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:03.215159 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:03.215159 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:03.215159 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:03.215159 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:03.215159 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:07:03.215159 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:07:03.215159 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:07:03.216063 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:07:03.216063 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:03.216063 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:03.216063 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:03.216063 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:03.216063 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:03.216063 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:03.216063 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:03.216063 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:03.216063 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:03.216063 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:03.216063 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:03.216063 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:03.216063 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:03.216063 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:03.216063 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:03.216063 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:03.216977 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:07:03.216977 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:07:03.216977 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:07:03.216977 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:03.216977 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 391/801] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:07:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:03.217971 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9343186c-7b86-42a3-b8ec-bb0e0614d034 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:03.253995 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e03967f-9380-4269-98d5-9da02698e25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:03.254656 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e03967f-9380-4269-98d5-9da02698e25a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:03.255546 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e03967f-9380-4269-98d5-9da02698e25a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:03.255546 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e03967f-9380-4269-98d5-9da02698e25a None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:03.255918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e03967f-9380-4269-98d5-9da02698e25a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:03.256225 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e03967f-9380-4269-98d5-9da02698e25a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:03.257652 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e03967f-9380-4269-98d5-9da02698e25a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:03.270069 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 411/802] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:07:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:03.271197 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e03967f-9380-4269-98d5-9da02698e25a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:03.282528 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:03.339694 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9343186c-7b86-42a3-b8ec-bb0e0614d034 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:03.374986 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e03967f-9380-4269-98d5-9da02698e25a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:03.376496 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996f89b5-c0ce-4812-9909-e3152e6a570e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:03.377343 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996f89b5-c0ce-4812-9909-e3152e6a570e None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:03.377628 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996f89b5-c0ce-4812-9909-e3152e6a570e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:03.377907 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996f89b5-c0ce-4812-9909-e3152e6a570e None admin] PATH_INFO: `/v3/services/network` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:03.378559 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996f89b5-c0ce-4812-9909-e3152e6a570e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:03.379935 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996f89b5-c0ce-4812-9909-e3152e6a570e None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:03.392348 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-996f89b5-c0ce-4812-9909-e3152e6a570e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 04 09:07:03.392348 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:03.392348 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:07:03.392348 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:07:03.392348 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:07:03.392348 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:07:03.392348 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:07:03.392348 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:07:03.392348 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 04 09:07:03.392348 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:03.392348 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:07:03.392348 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:03.392348 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:03.392348 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:03.392348 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:03.392348 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:03.392348 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:03.393417 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:03.393417 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:03.393417 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:03.393417 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:03.393417 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:03.393417 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:03.393417 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:07:03.393417 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:07:03.393417 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:07:03.393417 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:07:03.393417 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:03.393417 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:03.393417 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:03.393417 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:03.393417 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:03.393417 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:03.394336 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:03.394336 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:03.394336 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:03.394336 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:03.394336 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:03.394336 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:03.394336 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:03.394336 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:03.394336 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:03.394336 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:03.394336 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:07:03.394336 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:07:03.394336 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 04 09:07:03.394336 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:03.394336 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 392/803] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 4 09:07:03 2022] GET /identity/v3/services/network => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:03.395147 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996f89b5-c0ce-4812-9909-e3152e6a570e None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:03.415734 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83900308-9024-4513-9fe8-19420ce4c162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:03.416405 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83900308-9024-4513-9fe8-19420ce4c162 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:03.416690 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83900308-9024-4513-9fe8-19420ce4c162 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:03.416970 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83900308-9024-4513-9fe8-19420ce4c162 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:03.417724 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83900308-9024-4513-9fe8-19420ce4c162 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:03.418031 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83900308-9024-4513-9fe8-19420ce4c162 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:03.419377 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83900308-9024-4513-9fe8-19420ce4c162 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:03.423571 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:03.431445 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 412/804] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:07:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:03.432238 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83900308-9024-4513-9fe8-19420ce4c162 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:03.496825 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:03.504717 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996f89b5-c0ce-4812-9909-e3152e6a570e None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:03.530753 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83900308-9024-4513-9fe8-19420ce4c162 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:03.538939 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69808c9f-a81d-4198-a37e-db3619c0a290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:03.539608 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69808c9f-a81d-4198-a37e-db3619c0a290 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:03.539895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69808c9f-a81d-4198-a37e-db3619c0a290 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:03.540204 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69808c9f-a81d-4198-a37e-db3619c0a290 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:03.540933 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69808c9f-a81d-4198-a37e-db3619c0a290 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:03.541275 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69808c9f-a81d-4198-a37e-db3619c0a290 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:03.542673 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69808c9f-a81d-4198-a37e-db3619c0a290 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:03.554772 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 393/805] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:07:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:03.556015 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a37959-2abe-484c-b541-d1e7fca675c4 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:03.556302 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a37959-2abe-484c-b541-d1e7fca675c4 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:03.556581 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a37959-2abe-484c-b541-d1e7fca675c4 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:03.557823 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 394/806] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:07:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:03.558505 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a37959-2abe-484c-b541-d1e7fca675c4 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:03.569149 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f826a57-a4d6-467a-9055-770be0d62714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:03.570735 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f826a57-a4d6-467a-9055-770be0d62714 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:03.571032 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f826a57-a4d6-467a-9055-770be0d62714 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:03.571314 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f826a57-a4d6-467a-9055-770be0d62714 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:03.579072 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f826a57-a4d6-467a-9055-770be0d62714 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:03.580500 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f826a57-a4d6-467a-9055-770be0d62714 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:03.594806 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-8f826a57-a4d6-467a-9055-770be0d62714 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 04 09:07:03.594806 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:03.594806 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:03.594806 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:03.594806 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:03.594806 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:03.594806 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:03.594806 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:03.594806 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:03.594806 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:03.594806 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:03.594806 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:03.594806 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:07:03.594806 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:07:03.594806 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:07:03.594806 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:07:03.595940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:03.595940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:03.595940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:07:03.595940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:07:03.595940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:07:03.595940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:07:03.595940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:03.595940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:03.595940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:03.595940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:03.595940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:03.595940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:03.595940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:03.595940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:03.595940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:03.595940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:03.595940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:03.595940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:03.596913 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:03.596913 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:03.596913 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:07:03.596913 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:07:03.596913 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:07:03.596913 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:07:03.596913 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:07:03.596913 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:07:03.596913 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 04 09:07:03.596913 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:07:03.596913 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 413/807] 10.222.0.71 () {56 vars in 1093 bytes} [Mon Jul 4 09:07:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:03.596913 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f826a57-a4d6-467a-9055-770be0d62714 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:03.645014 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:03.661561 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a37959-2abe-484c-b541-d1e7fca675c4 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:03.696912 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a187e402-0dde-4969-8745-ec70bed811ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:03.697602 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187e402-0dde-4969-8745-ec70bed811ae None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:03.697882 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187e402-0dde-4969-8745-ec70bed811ae None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:03.698162 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187e402-0dde-4969-8745-ec70bed811ae None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:03.698857 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a187e402-0dde-4969-8745-ec70bed811ae None admin] RBAC: Adding query filter params (name=network) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:03.699163 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a187e402-0dde-4969-8745-ec70bed811ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:03.700526 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a187e402-0dde-4969-8745-ec70bed811ae None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:03.700822 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f826a57-a4d6-467a-9055-770be0d62714 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:03.704906 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 395/808] 10.222.0.71 () {56 vars in 1112 bytes} [Mon Jul 4 09:07:03 2022] GET /identity/v3/services?name=network => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:03.705699 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a187e402-0dde-4969-8745-ec70bed811ae None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:03.728905 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:03.737442 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce07f99-13f9-4a38-ac3c-e7f0da7b1673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:03.738234 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce07f99-13f9-4a38-ac3c-e7f0da7b1673 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:03.738520 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce07f99-13f9-4a38-ac3c-e7f0da7b1673 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:03.738803 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce07f99-13f9-4a38-ac3c-e7f0da7b1673 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:03.746217 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce07f99-13f9-4a38-ac3c-e7f0da7b1673 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:03.747652 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce07f99-13f9-4a38-ac3c-e7f0da7b1673 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:03.762391 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-3ce07f99-13f9-4a38-ac3c-e7f0da7b1673 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:07:03.762391 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:03.762391 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:03.762391 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:03.762391 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:03.762391 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:03.762391 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:03.762391 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:03.762391 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:03.762391 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:03.762391 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:03.762391 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:03.762391 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:07:03.762391 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:07:03.762391 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:07:03.763323 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:07:03.763323 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:03.763323 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:03.763323 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:03.763323 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:03.763323 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:03.763323 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:03.763323 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:03.763323 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:03.763323 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:03.763323 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:03.763323 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:03.763323 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:03.763323 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:03.763323 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:03.763323 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:03.763323 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:03.764297 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:07:03.764297 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:07:03.764297 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:07:03.764297 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:07:03.764297 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:07:03.764297 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:07:03.764297 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:07:03.764297 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:07:03.764297 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 414/809] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 4 09:07:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:03.765710 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a1c33-81d6-4af7-bf48-1d23378e1ee2 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:03.766001 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a1c33-81d6-4af7-bf48-1d23378e1ee2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:03.766283 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a1c33-81d6-4af7-bf48-1d23378e1ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:03.782617 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-313a1c33-81d6-4af7-bf48-1d23378e1ee2 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:03.783818 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-313a1c33-81d6-4af7-bf48-1d23378e1ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:03.808224 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a187e402-0dde-4969-8745-ec70bed811ae None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:03.841808 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea787eb-fd8b-4cbf-b28f-a729a8e95109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:03.842619 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea787eb-fd8b-4cbf-b28f-a729a8e95109 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:03.842904 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea787eb-fd8b-4cbf-b28f-a729a8e95109 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:03.843176 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea787eb-fd8b-4cbf-b28f-a729a8e95109 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:03.850073 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea787eb-fd8b-4cbf-b28f-a729a8e95109 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:03.850207 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 415/810] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:07:03.850876 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313a1c33-81d6-4af7-bf48-1d23378e1ee2 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:03.851606 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea787eb-fd8b-4cbf-b28f-a729a8e95109 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:03.875884 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-cea787eb-fd8b-4cbf-b28f-a729a8e95109 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 09:07:03.875884 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:03.875884 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:03.875884 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:03.875884 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:03.875884 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:03.875884 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:03.875884 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:03.875884 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:03.875884 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:03.875884 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:03.875884 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:03.875884 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:07:03.875884 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:07:03.875884 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:07:03.875884 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:07:03.876895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:03.876895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:03.876895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:07:03.876895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:07:03.876895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:07:03.876895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:07:03.876895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:03.876895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:03.876895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:03.876895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:03.876895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:03.876895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:03.876895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:03.876895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:03.876895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:03.876895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:03.876895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:03.876895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:03.877910 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:03.877910 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:03.877910 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:07:03.877910 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:07:03.877910 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:07:03.877910 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:07:03.877910 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:07:03.877910 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:07:03.877910 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 09:07:03.877910 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:03.877910 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 396/811] 10.222.0.71 () {56 vars in 1117 bytes} [Mon Jul 4 09:07:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:03.878537 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea787eb-fd8b-4cbf-b28f-a729a8e95109 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:03.890502 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:03.956067 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313a1c33-81d6-4af7-bf48-1d23378e1ee2 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:03.973524 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:03.979587 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea787eb-fd8b-4cbf-b28f-a729a8e95109 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:04.000107 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6626435f-bb2a-4dcf-a821-5d97c3450e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:04.000871 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626435f-bb2a-4dcf-a821-5d97c3450e18 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:04.001188 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626435f-bb2a-4dcf-a821-5d97c3450e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:04.001470 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626435f-bb2a-4dcf-a821-5d97c3450e18 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:04.002687 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6626435f-bb2a-4dcf-a821-5d97c3450e18 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:04.003441 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6626435f-bb2a-4dcf-a821-5d97c3450e18 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:04.004898 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6626435f-bb2a-4dcf-a821-5d97c3450e18 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:04.021414 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66326dc-0a9a-4957-a6af-ba2c02938904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:04.022085 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66326dc-0a9a-4957-a6af-ba2c02938904 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:04.022371 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66326dc-0a9a-4957-a6af-ba2c02938904 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:04.022650 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66326dc-0a9a-4957-a6af-ba2c02938904 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:04.023398 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66326dc-0a9a-4957-a6af-ba2c02938904 None admin] RBAC: Adding query filter params (type=network) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:04.023702 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66326dc-0a9a-4957-a6af-ba2c02938904 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:04.024401 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 416/812] 10.222.0.71 () {56 vars in 1133 bytes} [Mon Jul 4 09:07:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:04.025018 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6626435f-bb2a-4dcf-a821-5d97c3450e18 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:04.025264 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66326dc-0a9a-4957-a6af-ba2c02938904 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:04.032596 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 397/813] 10.222.0.71 () {56 vars in 1112 bytes} [Mon Jul 4 09:07:03 2022] GET /identity/v3/services?type=network => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:04.033254 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66326dc-0a9a-4957-a6af-ba2c02938904 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:04.108393 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:04.127949 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6626435f-bb2a-4dcf-a821-5d97c3450e18 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:04.133205 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66326dc-0a9a-4957-a6af-ba2c02938904 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:04.161661 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b01227-2ecd-475d-8214-e0fc1d7da87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:04.162321 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b01227-2ecd-475d-8214-e0fc1d7da87c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:04.162606 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b01227-2ecd-475d-8214-e0fc1d7da87c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:04.162885 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b01227-2ecd-475d-8214-e0fc1d7da87c None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:04.164334 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b01227-2ecd-475d-8214-e0fc1d7da87c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:04.164747 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b01227-2ecd-475d-8214-e0fc1d7da87c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:04.166239 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b01227-2ecd-475d-8214-e0fc1d7da87c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:04.168350 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7553b6-3901-456a-9c53-53d00b5005c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:04.169188 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7553b6-3901-456a-9c53-53d00b5005c4 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:04.169472 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7553b6-3901-456a-9c53-53d00b5005c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:04.169750 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7553b6-3901-456a-9c53-53d00b5005c4 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:04.170422 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7553b6-3901-456a-9c53-53d00b5005c4 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:04.171764 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7553b6-3901-456a-9c53-53d00b5005c4 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:04.177733 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 417/814] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 4 09:07:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:04.178357 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b01227-2ecd-475d-8214-e0fc1d7da87c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:04.185084 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-3b7553b6-3901-456a-9c53-53d00b5005c4 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 04 09:07:04.185084 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:04.185084 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:07:04.185084 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:07:04.185084 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:07:04.185084 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:07:04.185084 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:07:04.185084 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:07:04.185084 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 04 09:07:04.185084 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:04.185084 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:07:04.185084 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:04.185084 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:04.185084 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:04.185084 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:04.185084 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:04.186152 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:04.186152 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:04.186152 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:04.186152 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:04.186152 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:04.186152 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:04.186152 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:04.186152 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:07:04.186152 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:07:04.186152 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:07:04.186152 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:07:04.186152 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:04.186152 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:04.186152 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:04.186152 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:04.186152 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:04.187006 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:04.187006 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:04.187006 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:04.187006 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:04.187006 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:04.187006 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:04.187006 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:04.187006 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:04.187006 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:04.187006 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:04.187006 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:04.187006 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:07:04.187006 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:07:04.187006 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 04 09:07:04.187006 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:04.187006 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 398/815] 10.222.0.71 () {56 vars in 1129 bytes} [Mon Jul 4 09:07:04 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:04.187861 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7553b6-3901-456a-9c53-53d00b5005c4 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:04.217821 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:04.282380 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b01227-2ecd-475d-8214-e0fc1d7da87c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:04.295433 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7553b6-3901-456a-9c53-53d00b5005c4 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:04.316936 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208e6ca2-064a-47f2-a880-2f5b9ba9268d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:04.317635 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e6ca2-064a-47f2-a880-2f5b9ba9268d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:04.317915 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e6ca2-064a-47f2-a880-2f5b9ba9268d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:04.318192 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e6ca2-064a-47f2-a880-2f5b9ba9268d None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:04.319020 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208e6ca2-064a-47f2-a880-2f5b9ba9268d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:04.319331 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208e6ca2-064a-47f2-a880-2f5b9ba9268d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:04.320884 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208e6ca2-064a-47f2-a880-2f5b9ba9268d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:04.331473 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012348bc-1289-4349-9d3a-38d66426b0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:04.332913 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012348bc-1289-4349-9d3a-38d66426b0bc None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:04.333246 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012348bc-1289-4349-9d3a-38d66426b0bc None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:04.333524 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012348bc-1289-4349-9d3a-38d66426b0bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:04.338451 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 418/816] 10.222.0.71 () {56 vars in 1149 bytes} [Mon Jul 4 09:07:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:04.339247 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208e6ca2-064a-47f2-a880-2f5b9ba9268d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:04.345558 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:04.410524 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012348bc-1289-4349-9d3a-38d66426b0bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:04.411987 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012348bc-1289-4349-9d3a-38d66426b0bc None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:04.435954 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:04.442761 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208e6ca2-064a-47f2-a880-2f5b9ba9268d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:04.477349 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e6f72-d28b-44a9-876a-0771d7f8caae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:04.478013 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e6f72-d28b-44a9-876a-0771d7f8caae None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:04.478303 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e6f72-d28b-44a9-876a-0771d7f8caae None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:04.478586 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e6f72-d28b-44a9-876a-0771d7f8caae None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:04.479331 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31e6f72-d28b-44a9-876a-0771d7f8caae None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:04.480697 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31e6f72-d28b-44a9-876a-0771d7f8caae None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:04.501085 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 419/817] 10.222.0.71 () {60 vars in 1144 bytes} [Mon Jul 4 09:07:04 2022] POST /identity/v3/endpoints => generated 327 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:07:04.501891 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e6f72-d28b-44a9-876a-0771d7f8caae None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:04.515217 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 399/818] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:07:04 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:07:04.516470 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012348bc-1289-4349-9d3a-38d66426b0bc None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:04.564291 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:04.627545 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e6f72-d28b-44a9-876a-0771d7f8caae None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:04.639972 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012348bc-1289-4349-9d3a-38d66426b0bc None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:04.669261 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8913f4c-c14e-471d-9bfd-e3007a7eb094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:04.670495 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8913f4c-c14e-471d-9bfd-e3007a7eb094 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:04.670495 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8913f4c-c14e-471d-9bfd-e3007a7eb094 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:04.670837 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8913f4c-c14e-471d-9bfd-e3007a7eb094 None admin] PATH_INFO: `/v3/projects/4076e316243946e980ce47386b18b28f/users/88b41b55769a4c449716c3590ed58225/roles/f36e053b442540f08041ac7878760f65` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:04.672811 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8913f4c-c14e-471d-9bfd-e3007a7eb094 None admin] RBAC: Authorizing `identity:create_grant(project_id=4076e316243946e980ce47386b18b28f, user_id=88b41b55769a4c449716c3590ed58225, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:04.674413 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8913f4c-c14e-471d-9bfd-e3007a7eb094 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:04.681690 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2128a4-8ef0-4883-844e-fe062ff6ce93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:04.682438 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2128a4-8ef0-4883-844e-fe062ff6ce93 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:04.682743 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2128a4-8ef0-4883-844e-fe062ff6ce93 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:04.683041 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2128a4-8ef0-4883-844e-fe062ff6ce93 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:04.684207 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2128a4-8ef0-4883-844e-fe062ff6ce93 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:04.684483 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2128a4-8ef0-4883-844e-fe062ff6ce93 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:04.685879 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2128a4-8ef0-4883-844e-fe062ff6ce93 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:04.691180 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 400/820] 10.222.0.71 () {56 vars in 1124 bytes} [Mon Jul 4 09:07:04 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:04.691880 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2128a4-8ef0-4883-844e-fe062ff6ce93 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:04.697460 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 420/820] 10.222.0.71 () {58 vars in 1439 bytes} [Mon Jul 4 09:07:04 2022] PUT /identity/v3/projects/4076e316243946e980ce47386b18b28f/users/88b41b55769a4c449716c3590ed58225/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:07:04.699468 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8913f4c-c14e-471d-9bfd-e3007a7eb094 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:04.713771 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:04.836788 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2128a4-8ef0-4883-844e-fe062ff6ce93 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:04.839394 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:04.881837 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc96604-0d33-4aad-af60-9f38fe31efda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:04.883603 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc96604-0d33-4aad-af60-9f38fe31efda None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:04.883879 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc96604-0d33-4aad-af60-9f38fe31efda None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:04.884154 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc96604-0d33-4aad-af60-9f38fe31efda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:04.895927 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc96604-0d33-4aad-af60-9f38fe31efda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:04.897431 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc96604-0d33-4aad-af60-9f38fe31efda None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:04.919450 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-cbc96604-0d33-4aad-af60-9f38fe31efda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:07:04.919450 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:04.919450 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:04.919450 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:04.919450 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:04.919450 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:04.919450 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:04.919450 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:04.919450 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:04.919450 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:04.919450 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:04.919450 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:04.919450 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:07:04.919450 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:07:04.919450 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:07:04.920383 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:07:04.920383 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:04.920383 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:04.920383 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:04.920383 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:04.920383 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:04.920383 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:04.920383 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:04.920383 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:04.920383 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:04.920383 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:04.920383 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:04.920383 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:04.920383 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:04.920383 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:04.920383 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:04.920383 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:04.921710 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:07:04.921710 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:07:04.921710 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:07:04.921710 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:04.921710 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 401/821] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:07:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:04.925984 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc96604-0d33-4aad-af60-9f38fe31efda None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:04.957478 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8913f4c-c14e-471d-9bfd-e3007a7eb094 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:05.000560 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a92f8c4-9bde-421b-86cd-8b62aee30a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:05.002401 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a92f8c4-9bde-421b-86cd-8b62aee30a6f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:05.002705 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a92f8c4-9bde-421b-86cd-8b62aee30a6f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:05.002985 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a92f8c4-9bde-421b-86cd-8b62aee30a6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:05.019458 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a92f8c4-9bde-421b-86cd-8b62aee30a6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:05.020927 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a92f8c4-9bde-421b-86cd-8b62aee30a6f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:05.039629 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-9a92f8c4-9bde-421b-86cd-8b62aee30a6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:07:05.039629 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:05.039629 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:05.039629 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:05.039629 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:05.039629 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:05.039629 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:05.039629 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:05.039629 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:05.039629 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:05.039629 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:05.039629 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:05.039629 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:07:05.039629 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:07:05.039629 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:07:05.040564 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:07:05.040564 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:05.040564 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:05.040564 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:05.040564 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:05.040564 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:05.040564 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:05.040564 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:05.040564 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:05.040564 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:05.040564 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:05.040564 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:05.040564 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:05.040564 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:05.040564 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:05.040564 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:05.040564 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:05.041525 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:07:05.041525 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:07:05.041525 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:07:05.041525 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:07:05.041525 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 421/822] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:07:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:05.041804 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a92f8c4-9bde-421b-86cd-8b62aee30a6f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:05.047544 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc96604-0d33-4aad-af60-9f38fe31efda None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:05.085425 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeb3281-357d-4af3-baad-e892254cbfb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:05.086235 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb3281-357d-4af3-baad-e892254cbfb4 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:05.086411 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb3281-357d-4af3-baad-e892254cbfb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:05.086697 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb3281-357d-4af3-baad-e892254cbfb4 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:05.087675 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeb3281-357d-4af3-baad-e892254cbfb4 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:05.087985 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeb3281-357d-4af3-baad-e892254cbfb4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:05.089395 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeb3281-357d-4af3-baad-e892254cbfb4 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:05.094250 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 402/823] 10.222.0.71 () {56 vars in 1124 bytes} [Mon Jul 4 09:07:04 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:05.095028 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeb3281-357d-4af3-baad-e892254cbfb4 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:05.114547 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:05.119151 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:05.167627 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a92f8c4-9bde-421b-86cd-8b62aee30a6f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:05.204047 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb71a7f6-a7a2-40da-a0d8-93c51fb656f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:05.204622 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb71a7f6-a7a2-40da-a0d8-93c51fb656f8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:05.205005 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb71a7f6-a7a2-40da-a0d8-93c51fb656f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:05.205313 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb71a7f6-a7a2-40da-a0d8-93c51fb656f8 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:05.206151 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb71a7f6-a7a2-40da-a0d8-93c51fb656f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:05.206456 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb71a7f6-a7a2-40da-a0d8-93c51fb656f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:05.207801 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb71a7f6-a7a2-40da-a0d8-93c51fb656f8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:05.208527 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeb3281-357d-4af3-baad-e892254cbfb4 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:05.224225 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 422/824] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:07:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:05.224225 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb71a7f6-a7a2-40da-a0d8-93c51fb656f8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:05.252427 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b0f07d-6ff5-4269-b991-fbf243d2dba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:05.253227 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b0f07d-6ff5-4269-b991-fbf243d2dba7 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:05.253520 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b0f07d-6ff5-4269-b991-fbf243d2dba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:05.253799 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b0f07d-6ff5-4269-b991-fbf243d2dba7 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:05.254534 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b0f07d-6ff5-4269-b991-fbf243d2dba7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:05.254841 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b0f07d-6ff5-4269-b991-fbf243d2dba7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:05.256225 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b0f07d-6ff5-4269-b991-fbf243d2dba7 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:05.269929 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 403/825] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:07:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:05.270750 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b0f07d-6ff5-4269-b991-fbf243d2dba7 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:05.330741 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb71a7f6-a7a2-40da-a0d8-93c51fb656f8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:05.348347 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:05.365661 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe0843-3a80-4021-850f-4f242fc4475d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:05.366323 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe0843-3a80-4021-850f-4f242fc4475d None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:05.366632 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe0843-3a80-4021-850f-4f242fc4475d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:05.366966 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe0843-3a80-4021-850f-4f242fc4475d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:05.367704 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbe0843-3a80-4021-850f-4f242fc4475d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:05.369061 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbe0843-3a80-4021-850f-4f242fc4475d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:05.372943 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:05.405777 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 423/826] 10.222.0.71 () {60 vars in 1144 bytes} [Mon Jul 4 09:07:05 2022] POST /identity/v3/endpoints => generated 346 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:07:05.405777 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe0843-3a80-4021-850f-4f242fc4475d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:05.412544 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b0f07d-6ff5-4269-b991-fbf243d2dba7 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:05.469508 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c421303-cea1-425a-8dab-15dee6328656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:05.470697 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c421303-cea1-425a-8dab-15dee6328656 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:05.471014 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c421303-cea1-425a-8dab-15dee6328656 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:05.471301 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c421303-cea1-425a-8dab-15dee6328656 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:05.481803 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c421303-cea1-425a-8dab-15dee6328656 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:05.484450 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c421303-cea1-425a-8dab-15dee6328656 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:05.508409 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-0c421303-cea1-425a-8dab-15dee6328656 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:07:05.508409 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:05.508409 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:05.508409 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:05.508409 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:05.508409 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:05.508409 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:05.508409 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:05.508409 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:05.508409 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:05.508409 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:05.508409 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:05.508409 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:07:05.508409 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:07:05.508409 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:07:05.509440 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:07:05.509440 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:05.509440 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:05.509440 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:05.509440 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:05.509440 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:05.509440 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:05.509440 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:05.509440 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:05.509440 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:05.509440 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:05.509440 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:05.509440 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:05.509440 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:05.509440 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:05.509440 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:05.509440 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:05.510373 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:07:05.510373 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:07:05.510373 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:07:05.510373 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:07:05.510373 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:07:05.510373 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:07:05.510373 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:07:05.510373 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:05.518096 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 404/827] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 4 09:07:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:05.519834 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c421303-cea1-425a-8dab-15dee6328656 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:05.531436 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe0843-3a80-4021-850f-4f242fc4475d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:05.575544 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3f9baf-1dd7-42f4-ad26-45e8f912d77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:05.576308 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3f9baf-1dd7-42f4-ad26-45e8f912d77e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:05.576592 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3f9baf-1dd7-42f4-ad26-45e8f912d77e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:05.576870 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3f9baf-1dd7-42f4-ad26-45e8f912d77e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:05.584771 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3f9baf-1dd7-42f4-ad26-45e8f912d77e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:05.587586 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3f9baf-1dd7-42f4-ad26-45e8f912d77e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:05.603109 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-0d3f9baf-1dd7-42f4-ad26-45e8f912d77e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:07:05.603109 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:05.603109 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:05.603109 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:05.603109 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:05.603109 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:05.603109 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:05.603109 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:05.603109 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:05.603109 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:05.603109 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:05.603109 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:05.603109 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:07:05.603109 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:07:05.603109 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:07:05.604059 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:07:05.604059 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:05.604059 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:05.604059 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:05.604059 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:05.604059 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:05.604059 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:05.604059 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:05.604059 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:05.604059 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:05.604059 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:05.604059 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:05.604059 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:05.604059 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:05.604059 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:05.604059 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:05.604059 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:05.604999 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:07:05.604999 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:07:05.604999 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:07:05.604999 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:07:05.604999 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:07:05.604999 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:07:05.604999 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:07:05.604999 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:07:05.604999 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 424/828] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 4 09:07:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:05.610001 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3f9baf-1dd7-42f4-ad26-45e8f912d77e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:05.650278 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:05.662961 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c421303-cea1-425a-8dab-15dee6328656 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:05.701328 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a904e496-fd98-4cac-8e9f-38745ad69303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:05.702214 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a904e496-fd98-4cac-8e9f-38745ad69303 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:05.702600 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a904e496-fd98-4cac-8e9f-38745ad69303 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:05.702984 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a904e496-fd98-4cac-8e9f-38745ad69303 None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:05.703927 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a904e496-fd98-4cac-8e9f-38745ad69303 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:05.704342 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a904e496-fd98-4cac-8e9f-38745ad69303 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:05.705950 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a904e496-fd98-4cac-8e9f-38745ad69303 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:05.707934 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:05.722149 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 405/829] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 4 09:07:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:05.727747 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a904e496-fd98-4cac-8e9f-38745ad69303 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:05.738097 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3f9baf-1dd7-42f4-ad26-45e8f912d77e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:05.774234 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626af958-3f23-418f-9769-3bc72781099c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:05.774731 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626af958-3f23-418f-9769-3bc72781099c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:05.775011 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626af958-3f23-418f-9769-3bc72781099c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:05.775285 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626af958-3f23-418f-9769-3bc72781099c None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:05.776076 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626af958-3f23-418f-9769-3bc72781099c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:05.776384 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626af958-3f23-418f-9769-3bc72781099c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:05.777824 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626af958-3f23-418f-9769-3bc72781099c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:05.790820 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 425/830] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 4 09:07:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:07:05.796007 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626af958-3f23-418f-9769-3bc72781099c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:05.856875 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a904e496-fd98-4cac-8e9f-38745ad69303 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:05.893995 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7b0a67-76d9-4e0a-acae-709efc6b2364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:05.895144 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7b0a67-76d9-4e0a-acae-709efc6b2364 None admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:05.895593 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7b0a67-76d9-4e0a-acae-709efc6b2364 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:05.896021 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7b0a67-76d9-4e0a-acae-709efc6b2364 None admin] PATH_INFO: `/v3/projects/4076e316243946e980ce47386b18b28f/users/78668b939c5847b1b73e9c28b5d8db7f/roles/5752a9bdfe4b482ab37836eba7bd536b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:05.912057 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7b0a67-76d9-4e0a-acae-709efc6b2364 None admin] RBAC: Authorizing `identity:create_grant(project_id=4076e316243946e980ce47386b18b28f, user_id=78668b939c5847b1b73e9c28b5d8db7f, role_id=5752a9bdfe4b482ab37836eba7bd536b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:05.913909 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7b0a67-76d9-4e0a-acae-709efc6b2364 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:05.919087 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:05.926906 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626af958-3f23-418f-9769-3bc72781099c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:05.940726 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 406/831] 10.222.0.71 () {58 vars in 1439 bytes} [Mon Jul 4 09:07:05 2022] PUT /identity/v3/projects/4076e316243946e980ce47386b18b28f/users/78668b939c5847b1b73e9c28b5d8db7f/roles/5752a9bdfe4b482ab37836eba7bd536b => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:07:05.973585 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b3a222-67d3-4b28-9e53-ae2d0d5d06a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:05.974175 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b3a222-67d3-4b28-9e53-ae2d0d5d06a8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:05.974500 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b3a222-67d3-4b28-9e53-ae2d0d5d06a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:05.974788 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b3a222-67d3-4b28-9e53-ae2d0d5d06a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:05.975688 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b3a222-67d3-4b28-9e53-ae2d0d5d06a8 None admin] RBAC: Adding query filter params (role.id=f36e053b442540f08041ac7878760f65, scope.project.id=4076e316243946e980ce47386b18b28f, user.id=15039da209ec445d93d49e2bd6d854eb) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:05.976293 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b3a222-67d3-4b28-9e53-ae2d0d5d06a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:05.977626 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b3a222-67d3-4b28-9e53-ae2d0d5d06a8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:05.984927 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 426/832] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 4 09:07:05 2022] GET /identity/v3/role_assignments?user.id=15039da209ec445d93d49e2bd6d854eb&scope.project.id=4076e316243946e980ce47386b18b28f&role.id=f36e053b442540f08041ac7878760f65 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:07:06.781496 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca69c3f-3df2-4bca-b901-a2f8358bfb25 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:06.782070 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca69c3f-3df2-4bca-b901-a2f8358bfb25 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:06.782070 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca69c3f-3df2-4bca-b901-a2f8358bfb25 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:06.783668 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 407/833] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:07:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:06.789199 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a420d-f275-4737-9daa-58d0c6ae206c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:06.789594 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a420d-f275-4737-9daa-58d0c6ae206c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:06.789793 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a420d-f275-4737-9daa-58d0c6ae206c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:06.806965 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-694a420d-f275-4737-9daa-58d0c6ae206c None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:06.808161 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-694a420d-f275-4737-9daa-58d0c6ae206c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:06.881140 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 427/834] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:06 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:07:06.898160 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca69c3f-3df2-4bca-b901-a2f8358bfb25 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:06.945905 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:07.003118 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca69c3f-3df2-4bca-b901-a2f8358bfb25 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:07.037142 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434aeeba-7c84-4b48-9ff9-e13a15f95865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:07.037865 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434aeeba-7c84-4b48-9ff9-e13a15f95865 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:07.038151 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434aeeba-7c84-4b48-9ff9-e13a15f95865 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:07.038950 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434aeeba-7c84-4b48-9ff9-e13a15f95865 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:07.053642 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434aeeba-7c84-4b48-9ff9-e13a15f95865 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:07:07.061961 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-434aeeba-7c84-4b48-9ff9-e13a15f95865 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:07:07.061961 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:07.061961 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:07.061961 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:07.061961 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:07.061961 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:07.061961 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:07.061961 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:07.061961 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:07.061961 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:07.061961 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:07.061961 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:07.061961 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:07:07.061961 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:07:07.061961 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:07:07.061961 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:07:07.062893 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:07:07.062893 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:07:07.062893 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:07:07.062893 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:07:07.062893 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:07.062893 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:07.062893 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:07.062893 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:07.062893 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:07.062893 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:07.062893 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:07.062893 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:07.062893 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:07.062893 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:07.062893 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:07.062893 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:07.063734 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:07.063734 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:07.063734 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:07.063734 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:07.063734 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:07:07.063734 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:07:07.063734 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:07:07.063734 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:07:07.063734 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:07:07.063734 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:07:07.063734 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:07:07.063734 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:07.063734 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 408/835] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 4 09:07:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:07.069328 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694a420d-f275-4737-9daa-58d0c6ae206c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:07.166914 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:07.180822 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694a420d-f275-4737-9daa-58d0c6ae206c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:07.221197 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e371f-0768-45b5-91d9-7f6cb1f98d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:07.221878 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e371f-0768-45b5-91d9-7f6cb1f98d4f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:07.222224 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e371f-0768-45b5-91d9-7f6cb1f98d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:07.222506 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e371f-0768-45b5-91d9-7f6cb1f98d4f None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:07.223372 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655e371f-0768-45b5-91d9-7f6cb1f98d4f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:07.223686 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655e371f-0768-45b5-91d9-7f6cb1f98d4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:07.225071 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655e371f-0768-45b5-91d9-7f6cb1f98d4f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:07.234442 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 428/836] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:07:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:07.242216 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434aeeba-7c84-4b48-9ff9-e13a15f95865 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:07.346797 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434aeeba-7c84-4b48-9ff9-e13a15f95865 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:07.349168 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:07.385724 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc7c63b-c7fb-4ed9-a8e0-9f2910942602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:07.386535 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc7c63b-c7fb-4ed9-a8e0-9f2910942602 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:07.386814 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc7c63b-c7fb-4ed9-a8e0-9f2910942602 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:07.387082 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc7c63b-c7fb-4ed9-a8e0-9f2910942602 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:07.398952 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc7c63b-c7fb-4ed9-a8e0-9f2910942602 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:07.400543 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc7c63b-c7fb-4ed9-a8e0-9f2910942602 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:07.418218 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-3bc7c63b-c7fb-4ed9-a8e0-9f2910942602 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:07:07.418218 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:07.418218 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:07.418218 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:07.418218 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:07.418218 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:07.418218 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:07.418218 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:07.418218 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:07.418218 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:07.418218 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:07.418218 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:07.418218 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:07:07.418218 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:07:07.418218 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:07:07.419138 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:07:07.419138 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:07.419138 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:07.419138 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:07.419138 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:07.419138 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:07.419138 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:07.419138 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:07.419138 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:07.419138 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:07.419138 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:07.419138 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:07.419138 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:07.419138 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:07.419138 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:07.419138 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:07.419138 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:07.420052 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:07:07.420052 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:07:07.420052 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:07:07.420052 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:07.420052 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 409/837] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:07:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:07.426145 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e371f-0768-45b5-91d9-7f6cb1f98d4f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:07.526969 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e371f-0768-45b5-91d9-7f6cb1f98d4f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:07.560883 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51136f1-b1c1-4204-87b5-12ff9f8edb8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:07.561584 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51136f1-b1c1-4204-87b5-12ff9f8edb8a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:07.561865 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51136f1-b1c1-4204-87b5-12ff9f8edb8a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:07.562146 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51136f1-b1c1-4204-87b5-12ff9f8edb8a None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:07.562863 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51136f1-b1c1-4204-87b5-12ff9f8edb8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:07.563170 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51136f1-b1c1-4204-87b5-12ff9f8edb8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:07.564525 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51136f1-b1c1-4204-87b5-12ff9f8edb8a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:07.573425 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b39f9d6-02e0-476f-b296-7bdf41698e6b None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:07.573717 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b39f9d6-02e0-476f-b296-7bdf41698e6b None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:07.574003 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b39f9d6-02e0-476f-b296-7bdf41698e6b None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:07.575619 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 410/838] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:07:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:07.581120 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 429/839] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:07:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:07.583551 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b877f9-2aa2-42d9-8473-1898b4167e20 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:07.583848 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b877f9-2aa2-42d9-8473-1898b4167e20 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:07.584136 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b877f9-2aa2-42d9-8473-1898b4167e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:07.586207 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51136f1-b1c1-4204-87b5-12ff9f8edb8a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:07.602893 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-f1b877f9-2aa2-42d9-8473-1898b4167e20 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:07.604460 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f1b877f9-2aa2-42d9-8473-1898b4167e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:07.629444 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:07.703934 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 411/840] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:07 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:07:07.706596 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51136f1-b1c1-4204-87b5-12ff9f8edb8a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:07.728034 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b877f9-2aa2-42d9-8473-1898b4167e20 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:07.751873 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bda447-feae-4886-a10f-75f41e63d222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:07.753017 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bda447-feae-4886-a10f-75f41e63d222 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:07.754033 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bda447-feae-4886-a10f-75f41e63d222 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:07.754570 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bda447-feae-4886-a10f-75f41e63d222 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:07.765673 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bda447-feae-4886-a10f-75f41e63d222 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:07.767697 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bda447-feae-4886-a10f-75f41e63d222 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:07.787392 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-e5bda447-feae-4886-a10f-75f41e63d222 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 04 09:07:07.787392 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:07.787392 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:07.787392 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:07.787392 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:07.787392 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:07.787392 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:07.787392 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:07.787392 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:07.787392 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:07.787392 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:07.787392 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:07.787392 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:07:07.787392 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:07:07.787392 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:07:07.787392 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:07:07.789031 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:07.789031 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:07.789031 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:07:07.789031 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:07:07.789031 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:07:07.789031 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:07:07.789031 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:07.789031 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:07.789031 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:07.789031 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:07.789031 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:07.789031 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:07.789031 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:07.789031 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:07.789031 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:07.789031 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:07.789031 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:07.789031 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:07.790690 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:07.790690 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:07.790690 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:07:07.790690 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:07:07.790690 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:07:07.790690 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:07:07.790690 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:07:07.790690 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:07:07.790690 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 04 09:07:07.790690 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:07:07.793126 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 430/841] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 4 09:07:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:07.797703 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bda447-feae-4886-a10f-75f41e63d222 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:07.834839 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:07.859569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b877f9-2aa2-42d9-8473-1898b4167e20 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:07.887021 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:07.901241 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8c5c46-66aa-4cbb-9048-63a4fb0814c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:07.902463 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8c5c46-66aa-4cbb-9048-63a4fb0814c2 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:07.902989 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8c5c46-66aa-4cbb-9048-63a4fb0814c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:07.903466 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8c5c46-66aa-4cbb-9048-63a4fb0814c2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:07.904464 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8c5c46-66aa-4cbb-9048-63a4fb0814c2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:07.906229 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8c5c46-66aa-4cbb-9048-63a4fb0814c2 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:07.919497 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-3e8c5c46-66aa-4cbb-9048-63a4fb0814c2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 04 09:07:07.919497 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:07.919497 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:07:07.919497 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:07:07.919497 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:07:07.919497 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:07:07.919497 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:07:07.919497 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:07:07.919497 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 04 09:07:07.919497 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:07.919497 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:07:07.919497 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:07.919497 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:07.919497 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:07.919497 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:07.919497 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:07.919497 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:07.920535 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:07.920535 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:07.920535 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:07.920535 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:07.920535 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:07.920535 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:07.920535 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:07:07.920535 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:07:07.920535 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:07:07.920535 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:07:07.920535 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:07.920535 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:07.920535 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:07.920535 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:07.920535 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:07.920535 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:07.921902 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:07.921902 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:07.921902 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:07.921902 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:07.921902 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:07.921902 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:07.921902 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:07.921902 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:07.921902 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:07.921902 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:07.921902 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:07:07.921902 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:07:07.921902 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 04 09:07:07.921902 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:07.922654 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bda447-feae-4886-a10f-75f41e63d222 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:07.929259 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 412/842] 10.222.0.71 () {56 vars in 1114 bytes} [Mon Jul 4 09:07:07 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:07.930586 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8c5c46-66aa-4cbb-9048-63a4fb0814c2 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:07.956273 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934aab6d-9312-41a4-9414-43ac0440e777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:07.956946 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934aab6d-9312-41a4-9414-43ac0440e777 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:07.957258 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934aab6d-9312-41a4-9414-43ac0440e777 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:07.957539 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934aab6d-9312-41a4-9414-43ac0440e777 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:07.958378 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934aab6d-9312-41a4-9414-43ac0440e777 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:07.958724 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934aab6d-9312-41a4-9414-43ac0440e777 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:07.960139 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934aab6d-9312-41a4-9414-43ac0440e777 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:07.983677 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 431/843] 10.222.0.71 () {56 vars in 1135 bytes} [Mon Jul 4 09:07:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:07:07.985267 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934aab6d-9312-41a4-9414-43ac0440e777 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:08.046930 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8c5c46-66aa-4cbb-9048-63a4fb0814c2 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:08.086782 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd10a27-1023-4d4f-bb84-265aa2bec93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:08.088131 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd10a27-1023-4d4f-bb84-265aa2bec93a None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:08.088669 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd10a27-1023-4d4f-bb84-265aa2bec93a None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:08.089241 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd10a27-1023-4d4f-bb84-265aa2bec93a None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:08.089856 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934aab6d-9312-41a4-9414-43ac0440e777 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:08.090405 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd10a27-1023-4d4f-bb84-265aa2bec93a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:08.090914 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd10a27-1023-4d4f-bb84-265aa2bec93a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:08.092696 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd10a27-1023-4d4f-bb84-265aa2bec93a None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:08.103253 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 413/844] 10.222.0.71 () {56 vars in 1114 bytes} [Mon Jul 4 09:07:07 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:08.104505 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd10a27-1023-4d4f-bb84-265aa2bec93a None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:08.118752 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:08.125986 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:08.128582 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cec2b50-7f59-4c13-aea7-1bfd2103dad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:08.129296 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec2b50-7f59-4c13-aea7-1bfd2103dad9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:08.129643 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec2b50-7f59-4c13-aea7-1bfd2103dad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:08.129926 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec2b50-7f59-4c13-aea7-1bfd2103dad9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:08.201983 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cec2b50-7f59-4c13-aea7-1bfd2103dad9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:08.203442 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cec2b50-7f59-4c13-aea7-1bfd2103dad9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:08.220556 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd10a27-1023-4d4f-bb84-265aa2bec93a None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:08.254778 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142cd685-0934-49ad-8e78-a192d433e787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:08.255445 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142cd685-0934-49ad-8e78-a192d433e787 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:08.255727 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142cd685-0934-49ad-8e78-a192d433e787 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:08.256015 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142cd685-0934-49ad-8e78-a192d433e787 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:08.256715 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142cd685-0934-49ad-8e78-a192d433e787 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:08.257019 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142cd685-0934-49ad-8e78-a192d433e787 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:08.258919 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142cd685-0934-49ad-8e78-a192d433e787 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:08.263427 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 414/845] 10.222.0.71 () {56 vars in 1114 bytes} [Mon Jul 4 09:07:08 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:08.265143 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c11efc-f03b-4b9f-ba9c-29078a99e0b2 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:08.265436 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c11efc-f03b-4b9f-ba9c-29078a99e0b2 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:08.265718 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c11efc-f03b-4b9f-ba9c-29078a99e0b2 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:08.270990 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 415/846] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:07:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:08.270990 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7d175-7995-4b10-961e-2df105ccd37f None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:08.270990 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7d175-7995-4b10-961e-2df105ccd37f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:08.270990 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7d175-7995-4b10-961e-2df105ccd37f None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:08.270990 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 416/847] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:07:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:08.273966 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb871c8-7945-4c0b-864b-42a88df2b8c2 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:08.274601 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb871c8-7945-4c0b-864b-42a88df2b8c2 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:08.275206 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb871c8-7945-4c0b-864b-42a88df2b8c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:08.297400 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-bcb871c8-7945-4c0b-864b-42a88df2b8c2 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:08.298648 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-bcb871c8-7945-4c0b-864b-42a88df2b8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:08.316709 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 432/848] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:07:07 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:07:08.323419 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7123d4c7-8595-4c45-a7c4-3b82abcb14a5 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:08.323762 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7123d4c7-8595-4c45-a7c4-3b82abcb14a5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:08.323996 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7123d4c7-8595-4c45-a7c4-3b82abcb14a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:08.345120 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7123d4c7-8595-4c45-a7c4-3b82abcb14a5 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:08.346652 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7123d4c7-8595-4c45-a7c4-3b82abcb14a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:08.372344 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:08.385496 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 417/849] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:08 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:07:08.386526 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb871c8-7945-4c0b-864b-42a88df2b8c2 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:08.404891 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:08.438183 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 433/850] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:08 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:07:08.439039 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7123d4c7-8595-4c45-a7c4-3b82abcb14a5 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:08.501913 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb871c8-7945-4c0b-864b-42a88df2b8c2 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:08.538795 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3724b63-2d44-497c-8e1c-7b200dbbe655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:08.539696 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3724b63-2d44-497c-8e1c-7b200dbbe655 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:08.539995 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3724b63-2d44-497c-8e1c-7b200dbbe655 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:08.540281 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3724b63-2d44-497c-8e1c-7b200dbbe655 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:08.548202 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7123d4c7-8595-4c45-a7c4-3b82abcb14a5 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:08.550453 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3724b63-2d44-497c-8e1c-7b200dbbe655 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:08.551919 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3724b63-2d44-497c-8e1c-7b200dbbe655 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:08.569526 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-b3724b63-2d44-497c-8e1c-7b200dbbe655 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:07:08.569526 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:08.569526 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:08.569526 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:08.569526 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:08.569526 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:08.569526 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:08.569526 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:08.569526 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:08.569526 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:08.569526 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:08.569526 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:08.569526 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:07:08.569526 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:07:08.569526 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:07:08.571100 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:07:08.571100 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:08.571100 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:08.571100 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:08.571100 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:08.571100 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:08.571100 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:08.571100 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:08.571100 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:08.571100 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:08.571100 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:08.571100 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:08.571100 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:08.571100 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:08.571100 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:08.571100 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:08.571100 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:08.572053 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:07:08.572053 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:07:08.572053 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:07:08.572053 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:08.576089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 418/851] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:07:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:08.576991 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3724b63-2d44-497c-8e1c-7b200dbbe655 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:08.585033 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604176d6-8d92-4dd0-a8d9-725ed139f62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:08.586927 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604176d6-8d92-4dd0-a8d9-725ed139f62e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:08.587472 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604176d6-8d92-4dd0-a8d9-725ed139f62e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:08.587756 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604176d6-8d92-4dd0-a8d9-725ed139f62e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:08.588769 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604176d6-8d92-4dd0-a8d9-725ed139f62e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:08.590527 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604176d6-8d92-4dd0-a8d9-725ed139f62e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:08.603128 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-604176d6-8d92-4dd0-a8d9-725ed139f62e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 04 09:07:08.603128 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:08.603128 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:07:08.603128 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:07:08.603128 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:07:08.603128 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:07:08.603128 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:07:08.603128 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:07:08.603128 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 04 09:07:08.603128 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:07:08.603128 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:07:08.603128 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:07:08.603128 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:08.603128 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:08.603128 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:08.603128 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:08.604229 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:08.604229 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:08.604229 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:08.604229 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:08.604229 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:08.604229 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:08.604229 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:08.604229 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:07:08.604229 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:07:08.604229 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:07:08.604229 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:07:08.604229 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:08.604229 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:08.604229 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:08.604229 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:08.604229 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:08.605251 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:08.605251 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:08.605251 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:08.605251 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:08.605251 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:08.605251 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:08.605251 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:08.605251 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:08.605251 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:08.605251 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:08.605251 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:08.605251 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:07:08.605251 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:07:08.605251 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 04 09:07:08.605251 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:07:08.607559 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 434/852] 10.222.0.71 () {56 vars in 1132 bytes} [Mon Jul 4 09:07:08 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:08.608480 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604176d6-8d92-4dd0-a8d9-725ed139f62e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:08.634349 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:08.650056 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:08.693591 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3724b63-2d44-497c-8e1c-7b200dbbe655 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:08.715696 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604176d6-8d92-4dd0-a8d9-725ed139f62e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:08.730771 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f246ffb-f224-4af4-b798-971c7d6a2029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:08.731404 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f246ffb-f224-4af4-b798-971c7d6a2029 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:08.731689 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f246ffb-f224-4af4-b798-971c7d6a2029 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:08.731966 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f246ffb-f224-4af4-b798-971c7d6a2029 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:08.742915 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f246ffb-f224-4af4-b798-971c7d6a2029 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:07:08.750602 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-6f246ffb-f224-4af4-b798-971c7d6a2029 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:07:08.750602 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:08.750602 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:08.750602 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:08.750602 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:08.750602 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:08.750602 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:08.750602 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:08.750602 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:08.750602 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:08.750602 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:08.750602 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:08.750602 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:07:08.750602 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:07:08.750602 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:07:08.750602 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:07:08.751566 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:07:08.751566 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:07:08.751566 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:07:08.751566 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:07:08.751566 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:08.751566 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:08.751566 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:08.751566 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:08.751566 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:08.751566 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:08.751566 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:08.751566 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:08.751566 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:08.751566 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:08.751566 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:08.751566 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:08.752427 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:08.752427 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:08.752427 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:08.752427 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:08.752427 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:07:08.752427 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:07:08.752427 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:07:08.752427 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:07:08.752427 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:07:08.752427 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:07:08.752427 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:07:08.752427 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:08.752427 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 419/853] 10.222.0.71 () {56 vars in 1102 bytes} [Mon Jul 4 09:07:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:08.753907 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f246ffb-f224-4af4-b798-971c7d6a2029 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:08.758657 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc73668-4089-4031-a442-170ea2147cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:08.759343 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc73668-4089-4031-a442-170ea2147cdc None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:08.759628 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc73668-4089-4031-a442-170ea2147cdc None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:08.759911 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc73668-4089-4031-a442-170ea2147cdc None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:08.760649 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc73668-4089-4031-a442-170ea2147cdc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:08.760969 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc73668-4089-4031-a442-170ea2147cdc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:08.762383 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc73668-4089-4031-a442-170ea2147cdc None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:08.775666 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 435/854] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:07:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:08.776602 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc73668-4089-4031-a442-170ea2147cdc None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:08.863459 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:08.876356 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f246ffb-f224-4af4-b798-971c7d6a2029 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:08.883651 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc73668-4089-4031-a442-170ea2147cdc None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:08.899094 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:08.915206 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1708c39f-f9a9-4903-9891-2ff6649c24b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:08.915880 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1708c39f-f9a9-4903-9891-2ff6649c24b0 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:08.916170 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1708c39f-f9a9-4903-9891-2ff6649c24b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:08.916459 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1708c39f-f9a9-4903-9891-2ff6649c24b0 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:08.917216 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1708c39f-f9a9-4903-9891-2ff6649c24b0 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:08.917531 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1708c39f-f9a9-4903-9891-2ff6649c24b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:08.918924 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1708c39f-f9a9-4903-9891-2ff6649c24b0 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:08.920149 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6d3c90-e636-4506-968d-acd768247f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:08.920999 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d3c90-e636-4506-968d-acd768247f22 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:08.921451 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d3c90-e636-4506-968d-acd768247f22 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:08.921888 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d3c90-e636-4506-968d-acd768247f22 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:08.922774 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6d3c90-e636-4506-968d-acd768247f22 None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:08.923230 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6d3c90-e636-4506-968d-acd768247f22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:08.923648 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 420/855] 10.222.0.71 () {56 vars in 1126 bytes} [Mon Jul 4 09:07:08 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:08.924433 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1708c39f-f9a9-4903-9891-2ff6649c24b0 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:08.925154 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6d3c90-e636-4506-968d-acd768247f22 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:08.934421 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 436/856] 10.222.0.71 () {56 vars in 1103 bytes} [Mon Jul 4 09:07:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:08.935359 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6d3c90-e636-4506-968d-acd768247f22 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:09.033573 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1708c39f-f9a9-4903-9891-2ff6649c24b0 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:09.055892 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6d3c90-e636-4506-968d-acd768247f22 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:09.071153 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80719c80-42c3-42fd-a073-6312bc679b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:09.071949 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80719c80-42c3-42fd-a073-6312bc679b31 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:09.072232 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80719c80-42c3-42fd-a073-6312bc679b31 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:09.072511 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80719c80-42c3-42fd-a073-6312bc679b31 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:09.080426 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80719c80-42c3-42fd-a073-6312bc679b31 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:09.081898 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80719c80-42c3-42fd-a073-6312bc679b31 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:09.092763 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5bb93b-ad59-45d1-9d97-32326fb37aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:09.093648 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5bb93b-ad59-45d1-9d97-32326fb37aa3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:09.094107 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5bb93b-ad59-45d1-9d97-32326fb37aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:09.094534 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5bb93b-ad59-45d1-9d97-32326fb37aa3 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:09.095416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5bb93b-ad59-45d1-9d97-32326fb37aa3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:09.095870 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5bb93b-ad59-45d1-9d97-32326fb37aa3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:09.096694 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-80719c80-42c3-42fd-a073-6312bc679b31 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:07:09.096694 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:09.096694 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:09.096694 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:09.096694 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:09.096694 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:09.096694 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:09.096694 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:09.096694 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:09.096694 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:09.096694 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:09.096694 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:09.096694 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:07:09.096694 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:07:09.096694 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:07:09.097786 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:07:09.097786 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:09.097786 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:09.097786 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:09.097786 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:09.097786 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:09.097786 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:09.097786 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:09.097786 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:09.097786 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:09.097786 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:09.097786 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:09.097786 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:09.097786 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:09.097786 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:09.097786 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:09.097786 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:09.098910 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:07:09.098910 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:07:09.098910 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:07:09.098910 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:07:09.098910 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:07:09.098910 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:07:09.098910 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:07:09.098910 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:09.098910 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 421/857] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 4 09:07:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:09.098910 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80719c80-42c3-42fd-a073-6312bc679b31 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:09.100719 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5bb93b-ad59-45d1-9d97-32326fb37aa3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:09.105802 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 437/858] 10.222.0.71 () {56 vars in 1126 bytes} [Mon Jul 4 09:07:08 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:09.106747 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5bb93b-ad59-45d1-9d97-32326fb37aa3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:09.129006 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:09.173489 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:09.241617 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80719c80-42c3-42fd-a073-6312bc679b31 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:09.267167 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5bb93b-ad59-45d1-9d97-32326fb37aa3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:09.286091 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5563b6b-154a-450b-a903-35bbe4ffa21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:09.286889 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5563b6b-154a-450b-a903-35bbe4ffa21d None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:09.287198 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5563b6b-154a-450b-a903-35bbe4ffa21d None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:09.287475 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5563b6b-154a-450b-a903-35bbe4ffa21d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:09.302437 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5563b6b-154a-450b-a903-35bbe4ffa21d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:09.303400 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5563b6b-154a-450b-a903-35bbe4ffa21d None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:09.321460 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c03d730-84ef-4bb1-88cf-af84182d4bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:09.322420 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c03d730-84ef-4bb1-88cf-af84182d4bcb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:09.322909 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c03d730-84ef-4bb1-88cf-af84182d4bcb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:09.323366 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c03d730-84ef-4bb1-88cf-af84182d4bcb None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:09.324333 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c03d730-84ef-4bb1-88cf-af84182d4bcb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:09.324823 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c03d730-84ef-4bb1-88cf-af84182d4bcb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:09.324975 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-e5563b6b-154a-450b-a903-35bbe4ffa21d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:07:09.324975 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:09.324975 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:09.324975 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:09.324975 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:09.324975 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:09.324975 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:09.324975 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:09.324975 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:09.324975 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:09.324975 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:09.324975 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:09.324975 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:07:09.324975 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:07:09.324975 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:07:09.325899 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:07:09.325899 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:09.325899 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:09.325899 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:09.325899 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:09.325899 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:09.325899 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:09.325899 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:09.325899 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:09.325899 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:09.325899 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:09.325899 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:09.325899 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:09.325899 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:09.325899 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:09.325899 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:09.325899 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:09.326812 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:07:09.326812 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:07:09.326812 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:07:09.326812 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:09.326812 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 422/859] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:07:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:09.328687 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c03d730-84ef-4bb1-88cf-af84182d4bcb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:09.331779 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5563b6b-154a-450b-a903-35bbe4ffa21d None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:09.343619 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 438/860] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 4 09:07:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:09.350577 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c03d730-84ef-4bb1-88cf-af84182d4bcb None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:09.441618 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:09.460951 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:09.468170 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5563b6b-154a-450b-a903-35bbe4ffa21d None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:09.475168 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c03d730-84ef-4bb1-88cf-af84182d4bcb None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:09.510010 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fc7833-e3db-4749-8861-e3582111b32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:09.510897 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc7833-e3db-4749-8861-e3582111b32c None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:09.511290 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc7833-e3db-4749-8861-e3582111b32c None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:09.511639 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc7833-e3db-4749-8861-e3582111b32c None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:09.512492 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fc7833-e3db-4749-8861-e3582111b32c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:09.512885 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fc7833-e3db-4749-8861-e3582111b32c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:09.514505 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fc7833-e3db-4749-8861-e3582111b32c None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:09.525863 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfdc60c-d391-4e2c-82b7-251e71d05188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:09.527060 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfdc60c-d391-4e2c-82b7-251e71d05188 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:09.527643 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfdc60c-d391-4e2c-82b7-251e71d05188 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:09.528076 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfdc60c-d391-4e2c-82b7-251e71d05188 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:09.529079 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfdc60c-d391-4e2c-82b7-251e71d05188 None admin] RBAC: Adding query filter params (role.id=f36e053b442540f08041ac7878760f65, scope.project.id=4076e316243946e980ce47386b18b28f, user.id=88b41b55769a4c449716c3590ed58225) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:09.529562 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfdc60c-d391-4e2c-82b7-251e71d05188 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:09.531117 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfdc60c-d391-4e2c-82b7-251e71d05188 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:09.533304 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 423/861] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:07:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:09.534267 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fc7833-e3db-4749-8861-e3582111b32c None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:09.544561 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 439/862] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 4 09:07:09 2022] GET /identity/v3/role_assignments?user.id=88b41b55769a4c449716c3590ed58225&scope.project.id=4076e316243946e980ce47386b18b28f&role.id=f36e053b442540f08041ac7878760f65 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:07:09.660461 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fc7833-e3db-4749-8861-e3582111b32c None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:09.698469 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed75d9ef-89c9-4c15-81eb-0796d14f4412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:09.699224 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed75d9ef-89c9-4c15-81eb-0796d14f4412 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:09.699507 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed75d9ef-89c9-4c15-81eb-0796d14f4412 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:09.699787 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed75d9ef-89c9-4c15-81eb-0796d14f4412 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:09.708694 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed75d9ef-89c9-4c15-81eb-0796d14f4412 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:09.710245 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed75d9ef-89c9-4c15-81eb-0796d14f4412 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:09.727913 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-ed75d9ef-89c9-4c15-81eb-0796d14f4412 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 09:07:09.727913 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:09.727913 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:09.727913 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:09.727913 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:09.727913 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:09.727913 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:09.727913 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:09.727913 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:09.727913 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:09.727913 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:09.727913 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:09.727913 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:07:09.727913 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:07:09.727913 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:07:09.727913 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:07:09.729090 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:09.729090 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:09.729090 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:07:09.729090 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:07:09.729090 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:07:09.729090 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:07:09.729090 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:09.729090 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:09.729090 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:09.729090 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:09.729090 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:09.729090 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:09.729090 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:09.729090 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:09.729090 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:09.729090 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:09.729090 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:09.729090 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:09.730285 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:09.730285 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:09.730285 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:07:09.730285 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:07:09.730285 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:07:09.730285 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:07:09.730285 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:07:09.730285 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:07:09.730285 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 09:07:09.730285 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:09.730285 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 424/863] 10.222.0.71 () {56 vars in 1117 bytes} [Mon Jul 4 09:07:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:09.736823 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfdc60c-d391-4e2c-82b7-251e71d05188 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:09.842777 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfdc60c-d391-4e2c-82b7-251e71d05188 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:09.873746 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:09.879108 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd0a79b-adb2-4273-b1a7-a5d2bbfd1e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:09.880028 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd0a79b-adb2-4273-b1a7-a5d2bbfd1e87 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:09.880478 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd0a79b-adb2-4273-b1a7-a5d2bbfd1e87 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:09.880917 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd0a79b-adb2-4273-b1a7-a5d2bbfd1e87 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:09.882895 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd0a79b-adb2-4273-b1a7-a5d2bbfd1e87 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:09.882895 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd0a79b-adb2-4273-b1a7-a5d2bbfd1e87 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:09.884672 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd0a79b-adb2-4273-b1a7-a5d2bbfd1e87 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:09.908706 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 440/864] 10.222.0.71 () {56 vars in 1149 bytes} [Mon Jul 4 09:07:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:09.910214 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed75d9ef-89c9-4c15-81eb-0796d14f4412 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:09.930760 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:10.017724 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed75d9ef-89c9-4c15-81eb-0796d14f4412 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:10.052745 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60217634-4b2b-4fac-a5c1-0581f06b7138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:10.053432 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60217634-4b2b-4fac-a5c1-0581f06b7138 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:10.053715 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60217634-4b2b-4fac-a5c1-0581f06b7138 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:10.054705 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60217634-4b2b-4fac-a5c1-0581f06b7138 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:10.126434 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60217634-4b2b-4fac-a5c1-0581f06b7138 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:10.127974 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60217634-4b2b-4fac-a5c1-0581f06b7138 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:10.145944 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:10.235161 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 425/865] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:07:09 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:07:10.236653 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd0a79b-adb2-4273-b1a7-a5d2bbfd1e87 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:10.342894 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd0a79b-adb2-4273-b1a7-a5d2bbfd1e87 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:10.364179 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a96ec96-3900-4893-b8f9-42a48b180212 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:10.364503 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a96ec96-3900-4893-b8f9-42a48b180212 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:10.364756 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a96ec96-3900-4893-b8f9-42a48b180212 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:10.366398 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 426/866] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:07:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:10.371497 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c3758b-c721-475e-bd88-aca8e57e529c None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:10.371789 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c3758b-c721-475e-bd88-aca8e57e529c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:10.372077 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c3758b-c721-475e-bd88-aca8e57e529c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:10.380788 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c23bac-2c52-423c-be49-80191d475427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:10.381909 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c23bac-2c52-423c-be49-80191d475427 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:10.382450 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c23bac-2c52-423c-be49-80191d475427 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:10.382929 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c23bac-2c52-423c-be49-80191d475427 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:10.389741 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-22c3758b-c721-475e-bd88-aca8e57e529c None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:10.390961 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-22c3758b-c721-475e-bd88-aca8e57e529c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:10.397065 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c23bac-2c52-423c-be49-80191d475427 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:10.397065 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c23bac-2c52-423c-be49-80191d475427 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:10.416949 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-a1c23bac-2c52-423c-be49-80191d475427 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:07:10.416949 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:10.416949 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:10.416949 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:10.416949 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:10.416949 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:10.416949 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:10.416949 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:10.416949 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:10.416949 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:10.416949 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:10.416949 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:10.416949 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:07:10.416949 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:07:10.416949 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:07:10.418994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:07:10.418994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:10.418994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:10.418994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:10.418994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:10.418994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:10.418994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:10.418994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:10.418994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:10.418994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:10.418994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:10.418994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:10.418994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:10.418994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:10.418994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:10.418994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:10.418994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:10.420303 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:07:10.420303 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:07:10.420303 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:07:10.420303 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:07:10.420303 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 441/867] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:07:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:10.430716 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c23bac-2c52-423c-be49-80191d475427 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:10.481122 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 427/868] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:10 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:07:10.488936 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:10.504493 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c3758b-c721-475e-bd88-aca8e57e529c None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:10.549684 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c23bac-2c52-423c-be49-80191d475427 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:10.555325 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:10.584568 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0789a66-05eb-42d2-b2ab-2673fe5cc347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:10.586239 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0789a66-05eb-42d2-b2ab-2673fe5cc347 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:10.586869 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0789a66-05eb-42d2-b2ab-2673fe5cc347 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:10.587742 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0789a66-05eb-42d2-b2ab-2673fe5cc347 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:10.588706 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0789a66-05eb-42d2-b2ab-2673fe5cc347 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:10.589266 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0789a66-05eb-42d2-b2ab-2673fe5cc347 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:10.590838 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0789a66-05eb-42d2-b2ab-2673fe5cc347 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:10.603600 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 442/869] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:07:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:10.609340 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0789a66-05eb-42d2-b2ab-2673fe5cc347 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:10.621017 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c3758b-c721-475e-bd88-aca8e57e529c None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:10.665511 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac2f6c-79f9-4095-a8bc-dc3491fccc7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:10.667045 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac2f6c-79f9-4095-a8bc-dc3491fccc7d None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:10.667523 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac2f6c-79f9-4095-a8bc-dc3491fccc7d None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:10.667987 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac2f6c-79f9-4095-a8bc-dc3491fccc7d None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:10.669128 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceac2f6c-79f9-4095-a8bc-dc3491fccc7d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:10.671369 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceac2f6c-79f9-4095-a8bc-dc3491fccc7d None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:10.697522 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 428/870] 10.222.0.71 () {60 vars in 1141 bytes} [Mon Jul 4 09:07:10 2022] POST /identity/v3/services => generated 251 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:07:10.740303 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0789a66-05eb-42d2-b2ab-2673fe5cc347 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:10.743222 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:10.793992 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c56780a-7f76-43a3-917c-9f46c22a79f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:10.794853 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56780a-7f76-43a3-917c-9f46c22a79f0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:10.795138 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56780a-7f76-43a3-917c-9f46c22a79f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:10.795413 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56780a-7f76-43a3-917c-9f46c22a79f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:10.803318 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c56780a-7f76-43a3-917c-9f46c22a79f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:10.805621 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c56780a-7f76-43a3-917c-9f46c22a79f0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:10.822207 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-4c56780a-7f76-43a3-917c-9f46c22a79f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:07:10.822207 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:10.822207 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:10.822207 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:10.822207 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:10.822207 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:10.822207 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:10.822207 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:10.822207 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:10.822207 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:10.822207 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:10.822207 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:10.822207 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:07:10.822207 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:07:10.822207 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:07:10.823176 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:07:10.823176 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:10.823176 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:10.823176 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:10.823176 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:10.823176 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:10.823176 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:10.823176 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:10.823176 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:10.823176 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:10.823176 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:10.823176 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:10.823176 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:10.823176 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:10.823176 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:10.823176 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:10.823176 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:10.824105 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:07:10.824105 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:07:10.824105 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:07:10.824105 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:07:10.824105 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:07:10.824105 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:07:10.824105 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:07:10.824105 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:07:10.828354 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 443/871] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 4 09:07:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:07:10.829821 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac2f6c-79f9-4095-a8bc-dc3491fccc7d None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:11.066643 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac2f6c-79f9-4095-a8bc-dc3491fccc7d None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:11.076066 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:11.106424 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e79ad2-e377-4f22-853d-8123b763c403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:11.107470 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e79ad2-e377-4f22-853d-8123b763c403 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:11.107961 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e79ad2-e377-4f22-853d-8123b763c403 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:11.108497 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e79ad2-e377-4f22-853d-8123b763c403 None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:11.109528 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e79ad2-e377-4f22-853d-8123b763c403 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:11.110046 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e79ad2-e377-4f22-853d-8123b763c403 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:11.111711 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e79ad2-e377-4f22-853d-8123b763c403 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:11.125205 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 429/872] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 4 09:07:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:11.133068 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c56780a-7f76-43a3-917c-9f46c22a79f0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:11.243432 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c56780a-7f76-43a3-917c-9f46c22a79f0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:11.281625 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb119ba-a3ad-4165-a34a-b35cd203116f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:11.282747 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb119ba-a3ad-4165-a34a-b35cd203116f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:11.283256 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb119ba-a3ad-4165-a34a-b35cd203116f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:11.283737 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb119ba-a3ad-4165-a34a-b35cd203116f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:11.284780 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb119ba-a3ad-4165-a34a-b35cd203116f None admin] RBAC: Adding query filter params (role.id=5752a9bdfe4b482ab37836eba7bd536b, scope.project.id=4076e316243946e980ce47386b18b28f, user.id=78668b939c5847b1b73e9c28b5d8db7f) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:11.285355 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb119ba-a3ad-4165-a34a-b35cd203116f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:11.286995 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb119ba-a3ad-4165-a34a-b35cd203116f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:11.294517 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 444/873] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 4 09:07:11 2022] GET /identity/v3/role_assignments?user.id=78668b939c5847b1b73e9c28b5d8db7f&scope.project.id=4076e316243946e980ce47386b18b28f&role.id=5752a9bdfe4b482ab37836eba7bd536b => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:11.304327 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aeb9c4-2a81-49fb-a96b-7d1e126fab8f None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:11.304699 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aeb9c4-2a81-49fb-a96b-7d1e126fab8f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:11.305120 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aeb9c4-2a81-49fb-a96b-7d1e126fab8f None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:11.306958 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 430/874] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:07:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:11.312821 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f612864b-8a36-4098-83bb-d2e85014922d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:11.313215 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f612864b-8a36-4098-83bb-d2e85014922d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:11.313578 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f612864b-8a36-4098-83bb-d2e85014922d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:11.332237 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f612864b-8a36-4098-83bb-d2e85014922d None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:11.333673 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f612864b-8a36-4098-83bb-d2e85014922d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:11.355429 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:11.418578 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 445/875] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:11 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:07:11.442211 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aeb9c4-2a81-49fb-a96b-7d1e126fab8f None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:11.554955 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aeb9c4-2a81-49fb-a96b-7d1e126fab8f None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:11.591652 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbbc915-b0a7-4457-a516-7e2e58fee0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:11.592687 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbbc915-b0a7-4457-a516-7e2e58fee0fe None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:11.592976 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbbc915-b0a7-4457-a516-7e2e58fee0fe None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:11.593331 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbbc915-b0a7-4457-a516-7e2e58fee0fe None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:11.594074 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbbc915-b0a7-4457-a516-7e2e58fee0fe None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:11.595476 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbbc915-b0a7-4457-a516-7e2e58fee0fe None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:11.612634 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 431/876] 10.222.0.71 () {60 vars in 1141 bytes} [Mon Jul 4 09:07:11 2022] POST /identity/v3/services => generated 269 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:07:11.732007 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f646b1-840a-450d-be68-9ebeeeed8459 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:11.732307 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f646b1-840a-450d-be68-9ebeeeed8459 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:11.732573 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f646b1-840a-450d-be68-9ebeeeed8459 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:11.734287 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 446/877] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:07:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:11.739777 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb5af1-30d2-45ce-b0c0-25685ab5893f None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:11.740061 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb5af1-30d2-45ce-b0c0-25685ab5893f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:11.740348 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb5af1-30d2-45ce-b0c0-25685ab5893f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:11.759448 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-dfcb5af1-30d2-45ce-b0c0-25685ab5893f None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:11.760659 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-dfcb5af1-30d2-45ce-b0c0-25685ab5893f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:11.783971 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:11.841386 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 432/878] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:11 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:07:11.863575 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f646b1-840a-450d-be68-9ebeeeed8459 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:11.977988 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f646b1-840a-450d-be68-9ebeeeed8459 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:12.015547 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ef5fde-ba27-4f26-a2a3-4eb5b1947405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:12.016357 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef5fde-ba27-4f26-a2a3-4eb5b1947405 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:12.016639 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef5fde-ba27-4f26-a2a3-4eb5b1947405 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:12.016911 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef5fde-ba27-4f26-a2a3-4eb5b1947405 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:12.017595 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ef5fde-ba27-4f26-a2a3-4eb5b1947405 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:12.018965 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ef5fde-ba27-4f26-a2a3-4eb5b1947405 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:12.031830 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-d9ef5fde-ba27-4f26-a2a3-4eb5b1947405 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 09:07:12.031830 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:12.031830 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:07:12.031830 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:07:12.031830 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:07:12.031830 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:07:12.031830 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:07:12.031830 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:07:12.031830 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 09:07:12.031830 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:07:12.031830 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:07:12.031830 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:07:12.031830 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:12.031830 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:12.031830 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:12.031830 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:12.031830 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:12.032867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:12.032867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:12.032867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:12.032867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:12.032867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:12.032867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:12.032867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:07:12.032867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:07:12.032867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:07:12.032867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:07:12.032867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:12.032867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:12.032867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:12.032867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:12.032867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:12.032867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:12.033763 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:12.033763 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:12.033763 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:12.033763 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:12.033763 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:12.033763 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:12.033763 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:12.033763 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:12.033763 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:12.033763 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:12.033763 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:07:12.033763 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:07:12.033763 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 09:07:12.033763 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:07:12.033763 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 447/879] 10.222.0.71 () {56 vars in 1126 bytes} [Mon Jul 4 09:07:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:12.041134 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcb5af1-30d2-45ce-b0c0-25685ab5893f None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:12.146211 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcb5af1-30d2-45ce-b0c0-25685ab5893f None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:12.181984 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cebdf85-9e5d-4422-b3cf-c1c4a1cab91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:12.182660 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cebdf85-9e5d-4422-b3cf-c1c4a1cab91a None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:12.182942 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cebdf85-9e5d-4422-b3cf-c1c4a1cab91a None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:12.183222 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cebdf85-9e5d-4422-b3cf-c1c4a1cab91a None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:12.183941 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cebdf85-9e5d-4422-b3cf-c1c4a1cab91a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:12.184251 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cebdf85-9e5d-4422-b3cf-c1c4a1cab91a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:12.185741 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cebdf85-9e5d-4422-b3cf-c1c4a1cab91a None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:12.191069 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 433/880] 10.222.0.71 () {56 vars in 1122 bytes} [Mon Jul 4 09:07:12 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:12.197083 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ef5fde-ba27-4f26-a2a3-4eb5b1947405 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:12.219462 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:12.310894 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ef5fde-ba27-4f26-a2a3-4eb5b1947405 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:12.347234 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c546f6d5-bfaa-4a04-a9d2-3c3afadb4321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:12.347904 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c546f6d5-bfaa-4a04-a9d2-3c3afadb4321 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:12.348186 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c546f6d5-bfaa-4a04-a9d2-3c3afadb4321 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:12.348463 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c546f6d5-bfaa-4a04-a9d2-3c3afadb4321 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:12.349205 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c546f6d5-bfaa-4a04-a9d2-3c3afadb4321 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:12.349516 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c546f6d5-bfaa-4a04-a9d2-3c3afadb4321 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:12.350941 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c546f6d5-bfaa-4a04-a9d2-3c3afadb4321 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:12.355951 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 448/881] 10.222.0.71 () {56 vars in 1122 bytes} [Mon Jul 4 09:07:12 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:12.921808 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a143c0-c120-4e08-b744-8c2cac80a71e None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:12.922190 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a143c0-c120-4e08-b744-8c2cac80a71e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:12.922392 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a143c0-c120-4e08-b744-8c2cac80a71e None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:12.924122 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 434/882] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:07:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:12.929775 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852cbc5e-4b99-4350-9de2-10fd35f3b9e2 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:12.930220 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852cbc5e-4b99-4350-9de2-10fd35f3b9e2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:12.930510 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852cbc5e-4b99-4350-9de2-10fd35f3b9e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:12.947457 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-852cbc5e-4b99-4350-9de2-10fd35f3b9e2 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:12.948650 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-852cbc5e-4b99-4350-9de2-10fd35f3b9e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:13.034080 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 449/883] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:12 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:07:13.063878 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a143c0-c120-4e08-b744-8c2cac80a71e None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:13.083587 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:13.176262 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a143c0-c120-4e08-b744-8c2cac80a71e None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:13.213844 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7d5650-028d-4289-ab4b-f5fc55bdb3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:13.214643 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d5650-028d-4289-ab4b-f5fc55bdb3a2 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:13.214925 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d5650-028d-4289-ab4b-f5fc55bdb3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:13.215201 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d5650-028d-4289-ab4b-f5fc55bdb3a2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:13.215903 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7d5650-028d-4289-ab4b-f5fc55bdb3a2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:13.217358 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7d5650-028d-4289-ab4b-f5fc55bdb3a2 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:13.230152 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-de7d5650-028d-4289-ab4b-f5fc55bdb3a2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 04 09:07:13.230152 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:13.230152 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:07:13.230152 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:07:13.230152 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:07:13.230152 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:07:13.230152 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:07:13.230152 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:07:13.230152 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 04 09:07:13.230152 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:13.230152 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:07:13.230152 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:13.230152 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:13.230152 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:13.230152 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:13.230152 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:13.230152 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:13.232414 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:13.232414 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:13.232414 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:13.232414 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:13.232414 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:13.232414 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:13.232414 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:07:13.232414 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:07:13.232414 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:07:13.232414 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:07:13.232414 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:13.232414 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:13.232414 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:13.232414 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:13.232414 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:13.232414 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:13.234471 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:13.234471 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:13.234471 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:13.234471 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:13.234471 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:13.234471 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:13.234471 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:13.234471 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:13.234471 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:13.234471 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:13.234471 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:07:13.234471 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:07:13.234471 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 04 09:07:13.234471 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:13.234471 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 435/884] 10.222.0.71 () {56 vars in 1114 bytes} [Mon Jul 4 09:07:13 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:13.241059 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852cbc5e-4b99-4350-9de2-10fd35f3b9e2 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:13.250269 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:13.352013 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852cbc5e-4b99-4350-9de2-10fd35f3b9e2 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:13.393522 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea159865-07e2-4013-b22f-07db1895a810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:13.394209 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea159865-07e2-4013-b22f-07db1895a810 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:13.394495 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea159865-07e2-4013-b22f-07db1895a810 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:13.394776 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea159865-07e2-4013-b22f-07db1895a810 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:13.395503 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea159865-07e2-4013-b22f-07db1895a810 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:13.395872 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea159865-07e2-4013-b22f-07db1895a810 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:13.397352 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea159865-07e2-4013-b22f-07db1895a810 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:13.403098 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 450/885] 10.222.0.71 () {56 vars in 1114 bytes} [Mon Jul 4 09:07:13 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:13.408300 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7d5650-028d-4289-ab4b-f5fc55bdb3a2 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:13.509461 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:13.515953 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7d5650-028d-4289-ab4b-f5fc55bdb3a2 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:13.555352 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090476be-80fd-44f3-8639-bccdb65d1fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:13.556024 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090476be-80fd-44f3-8639-bccdb65d1fe6 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:13.556312 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090476be-80fd-44f3-8639-bccdb65d1fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:13.556589 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090476be-80fd-44f3-8639-bccdb65d1fe6 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:13.557324 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090476be-80fd-44f3-8639-bccdb65d1fe6 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:13.557632 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090476be-80fd-44f3-8639-bccdb65d1fe6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:13.559005 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090476be-80fd-44f3-8639-bccdb65d1fe6 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:13.564674 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 436/886] 10.222.0.71 () {56 vars in 1114 bytes} [Mon Jul 4 09:07:13 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:13.570592 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea159865-07e2-4013-b22f-07db1895a810 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:13.652687 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:13.674828 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b1dd0-8576-4051-bde5-ffac46f79826 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:13.675144 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b1dd0-8576-4051-bde5-ffac46f79826 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:13.675393 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b1dd0-8576-4051-bde5-ffac46f79826 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:13.676733 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 437/887] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:07:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:07:13.681060 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea159865-07e2-4013-b22f-07db1895a810 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:13.682792 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b1489b-f672-4d3d-9120-12bd8bf1050e None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:13.683083 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b1489b-f672-4d3d-9120-12bd8bf1050e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:13.683569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b1489b-f672-4d3d-9120-12bd8bf1050e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:13.708049 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-03b1489b-f672-4d3d-9120-12bd8bf1050e None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:13.710251 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-03b1489b-f672-4d3d-9120-12bd8bf1050e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:13.719043 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f0f43a-ebc6-4cdd-9c4e-3a241bc94ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:13.719708 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0f43a-ebc6-4cdd-9c4e-3a241bc94ebf None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:13.720001 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0f43a-ebc6-4cdd-9c4e-3a241bc94ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:13.720755 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0f43a-ebc6-4cdd-9c4e-3a241bc94ebf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:13.721608 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f0f43a-ebc6-4cdd-9c4e-3a241bc94ebf None admin] RBAC: Adding query filter params (interface=public, service_id=763f774b51da4d5badb51e29fe880a37, region_id=RegionOne) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:13.721922 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f0f43a-ebc6-4cdd-9c4e-3a241bc94ebf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:13.723728 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f0f43a-ebc6-4cdd-9c4e-3a241bc94ebf None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:13.734784 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 451/888] 10.222.0.71 () {56 vars in 1251 bytes} [Mon Jul 4 09:07:13 2022] GET /identity/v3/endpoints?service_id=763f774b51da4d5badb51e29fe880a37&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:13.741243 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f0f43a-ebc6-4cdd-9c4e-3a241bc94ebf None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:13.795914 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 438/889] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:13 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:07:13.797477 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bcdd98-efd8-4220-96b1-4ba19e69a966 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:13.797893 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bcdd98-efd8-4220-96b1-4ba19e69a966 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:13.798296 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bcdd98-efd8-4220-96b1-4ba19e69a966 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:13.799929 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 439/890] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:07:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:13.806360 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e5d9a1-93bd-4f7a-b093-7cc8739b58ce None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:13.807049 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e5d9a1-93bd-4f7a-b093-7cc8739b58ce None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:13.807907 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e5d9a1-93bd-4f7a-b093-7cc8739b58ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:13.828051 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-78e5d9a1-93bd-4f7a-b093-7cc8739b58ce None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:13.829645 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-78e5d9a1-93bd-4f7a-b093-7cc8739b58ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:13.856504 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f0f43a-ebc6-4cdd-9c4e-3a241bc94ebf None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:13.893847 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032bdea6-3d36-4c9f-85c6-650ad809a348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:13.894514 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032bdea6-3d36-4c9f-85c6-650ad809a348 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:13.894800 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032bdea6-3d36-4c9f-85c6-650ad809a348 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:13.895086 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032bdea6-3d36-4c9f-85c6-650ad809a348 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:13.895763 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032bdea6-3d36-4c9f-85c6-650ad809a348 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:13.897164 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032bdea6-3d36-4c9f-85c6-650ad809a348 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:13.913070 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 440/891] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:13 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:07:13.913826 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 452/892] 10.222.0.71 () {56 vars in 1087 bytes} [Mon Jul 4 09:07:13 2022] GET /identity/v3/services => generated 1268 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:07:13.913933 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e5d9a1-93bd-4f7a-b093-7cc8739b58ce None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:13.918352 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:13.933141 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032bdea6-3d36-4c9f-85c6-650ad809a348 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:13.936270 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:14.050446 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032bdea6-3d36-4c9f-85c6-650ad809a348 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:14.058931 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e5d9a1-93bd-4f7a-b093-7cc8739b58ce None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:14.094400 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa31aa80-07dc-4086-a7ab-c1a643c39c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:14.095756 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa31aa80-07dc-4086-a7ab-c1a643c39c97 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:14.096745 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa31aa80-07dc-4086-a7ab-c1a643c39c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:14.096745 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa31aa80-07dc-4086-a7ab-c1a643c39c97 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:14.097188 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa31aa80-07dc-4086-a7ab-c1a643c39c97 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:14.098593 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa31aa80-07dc-4086-a7ab-c1a643c39c97 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:14.104129 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e36046-d259-4ecc-b538-b3e3e5e67217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:14.104958 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e36046-d259-4ecc-b538-b3e3e5e67217 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:14.105272 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e36046-d259-4ecc-b538-b3e3e5e67217 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:14.105548 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e36046-d259-4ecc-b538-b3e3e5e67217 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:14.113940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-aa31aa80-07dc-4086-a7ab-c1a643c39c97 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 04 09:07:14.113940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:14.113940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:07:14.113940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:07:14.113940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:07:14.113940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:07:14.113940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:07:14.113940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:07:14.113940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 04 09:07:14.113940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:07:14.113940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:07:14.113940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:07:14.113940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:14.113940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:14.113940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:14.113940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:14.114911 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:14.114911 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:14.114911 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:14.114911 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:14.114911 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:14.114911 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:14.114911 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:14.114911 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:07:14.114911 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:07:14.114911 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:07:14.114911 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:07:14.114911 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:14.114911 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:14.114911 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:14.114911 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:14.114911 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:14.115768 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:14.115768 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:14.115768 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:14.115768 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:14.115768 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:14.115768 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:14.115768 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:14.115768 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:14.115768 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:14.115768 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:14.115768 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:14.115768 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:07:14.115768 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:07:14.115768 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 04 09:07:14.115768 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:07:14.117125 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 453/893] 10.222.0.71 () {56 vars in 1132 bytes} [Mon Jul 4 09:07:13 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:07:14.119402 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e36046-d259-4ecc-b538-b3e3e5e67217 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:07:14.125799 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa31aa80-07dc-4086-a7ab-c1a643c39c97 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:14.128197 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-49e36046-d259-4ecc-b538-b3e3e5e67217 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 09:07:14.128197 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:14.128197 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:14.128197 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:14.128197 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:14.128197 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:14.128197 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:14.128197 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:14.128197 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:14.128197 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:14.128197 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:14.128197 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:14.128197 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:07:14.128197 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:07:14.128197 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:07:14.128197 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:07:14.129435 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:07:14.129435 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:07:14.129435 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:07:14.129435 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:07:14.129435 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:14.129435 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:14.129435 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:14.129435 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:14.129435 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:14.129435 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:14.129435 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:14.129435 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:14.129435 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:14.129435 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:14.129435 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:14.129435 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:14.130470 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:14.130470 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:14.130470 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:14.130470 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:14.130470 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:07:14.130470 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:07:14.130470 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:07:14.130470 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:07:14.130470 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:07:14.130470 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:07:14.130470 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 09:07:14.130470 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:14.134534 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 441/894] 10.222.0.71 () {56 vars in 1120 bytes} [Mon Jul 4 09:07:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:14.141267 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e36046-d259-4ecc-b538-b3e3e5e67217 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:14.200608 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:14.237244 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:14.248119 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa31aa80-07dc-4086-a7ab-c1a643c39c97 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:14.264858 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e36046-d259-4ecc-b538-b3e3e5e67217 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:14.284393 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d549aa-352b-4147-b1b1-179fac8aab63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:14.285215 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d549aa-352b-4147-b1b1-179fac8aab63 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:14.285500 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d549aa-352b-4147-b1b1-179fac8aab63 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:14.285778 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d549aa-352b-4147-b1b1-179fac8aab63 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:14.286501 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d549aa-352b-4147-b1b1-179fac8aab63 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:14.286806 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d549aa-352b-4147-b1b1-179fac8aab63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:14.288182 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d549aa-352b-4147-b1b1-179fac8aab63 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:14.298161 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 454/895] 10.222.0.71 () {56 vars in 1126 bytes} [Mon Jul 4 09:07:14 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:14.299393 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d549aa-352b-4147-b1b1-179fac8aab63 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:14.302372 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b835582-4d87-4dc5-8a15-4daeb35755d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:14.303307 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b835582-4d87-4dc5-8a15-4daeb35755d8 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:14.303757 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b835582-4d87-4dc5-8a15-4daeb35755d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:14.304973 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b835582-4d87-4dc5-8a15-4daeb35755d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:14.305937 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b835582-4d87-4dc5-8a15-4daeb35755d8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:14.306400 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b835582-4d87-4dc5-8a15-4daeb35755d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:14.307931 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b835582-4d87-4dc5-8a15-4daeb35755d8 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:14.321174 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 442/896] 10.222.0.71 () {56 vars in 1115 bytes} [Mon Jul 4 09:07:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:07:14.323433 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b835582-4d87-4dc5-8a15-4daeb35755d8 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:14.409518 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d549aa-352b-4147-b1b1-179fac8aab63 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:14.428688 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b835582-4d87-4dc5-8a15-4daeb35755d8 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:14.440164 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:14.447219 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a1514d-c94a-4589-8cb2-bc9755403492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:14.447896 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a1514d-c94a-4589-8cb2-bc9755403492 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:14.448178 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a1514d-c94a-4589-8cb2-bc9755403492 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:14.448454 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a1514d-c94a-4589-8cb2-bc9755403492 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:14.449341 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a1514d-c94a-4589-8cb2-bc9755403492 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:14.449859 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a1514d-c94a-4589-8cb2-bc9755403492 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:14.451909 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a1514d-c94a-4589-8cb2-bc9755403492 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:14.456880 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 455/897] 10.222.0.71 () {56 vars in 1126 bytes} [Mon Jul 4 09:07:14 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:07:14.458308 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f22c6-2ec0-4318-8a1d-694d62ede7aa None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:14.458597 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f22c6-2ec0-4318-8a1d-694d62ede7aa None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:14.458876 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f22c6-2ec0-4318-8a1d-694d62ede7aa None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:14.460256 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 456/898] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:07:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:14.463415 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581f22c6-2ec0-4318-8a1d-694d62ede7aa None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:14.467874 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:14.478232 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a93239-d00c-4c87-86a5-d2b40682887c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:14.479041 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a93239-d00c-4c87-86a5-d2b40682887c None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:14.479310 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a93239-d00c-4c87-86a5-d2b40682887c None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:14.479586 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a93239-d00c-4c87-86a5-d2b40682887c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:14.490979 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a93239-d00c-4c87-86a5-d2b40682887c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:14.493021 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a93239-d00c-4c87-86a5-d2b40682887c None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:14.510514 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-86a93239-d00c-4c87-86a5-d2b40682887c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:07:14.510514 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:14.510514 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:14.510514 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:14.510514 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:14.510514 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:14.510514 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:14.510514 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:14.510514 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:14.510514 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:14.510514 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:14.510514 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:14.510514 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:07:14.510514 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:07:14.510514 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:07:14.511591 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:07:14.511591 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:14.511591 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:14.511591 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:14.511591 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:14.511591 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:14.511591 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:14.511591 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:14.511591 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:14.511591 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:14.511591 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:14.511591 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:14.511591 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:14.511591 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:14.511591 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:14.511591 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:14.511591 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:14.512599 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:07:14.512599 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:07:14.512599 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:07:14.512599 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:14.512599 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 443/899] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:07:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:07:14.512874 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ea0da-fdc9-47e5-9e41-2eafaba92f4d None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:14.513187 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ea0da-fdc9-47e5-9e41-2eafaba92f4d None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:14.513471 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ea0da-fdc9-47e5-9e41-2eafaba92f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:14.531105 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-cd1ea0da-fdc9-47e5-9e41-2eafaba92f4d None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:14.532306 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-cd1ea0da-fdc9-47e5-9e41-2eafaba92f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:14.578251 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581f22c6-2ec0-4318-8a1d-694d62ede7aa None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:14.607831 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 444/900] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:14 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:07:14.608901 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1ea0da-fdc9-47e5-9e41-2eafaba92f4d None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:14.635032 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dd8d0e-e5e5-40a9-b8f2-94b92331e738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:14.637169 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dd8d0e-e5e5-40a9-b8f2-94b92331e738 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:14.637755 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dd8d0e-e5e5-40a9-b8f2-94b92331e738 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:14.638219 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dd8d0e-e5e5-40a9-b8f2-94b92331e738 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:14.639605 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dd8d0e-e5e5-40a9-b8f2-94b92331e738 None admin] RBAC: Adding query filter params (interface=public, service_id=c4e9e15e1f634af38d6ea92c92d5d8cf, region_id=RegionOne) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:14.640129 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dd8d0e-e5e5-40a9-b8f2-94b92331e738 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:14.642533 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dd8d0e-e5e5-40a9-b8f2-94b92331e738 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:14.650807 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 457/901] 10.222.0.71 () {56 vars in 1251 bytes} [Mon Jul 4 09:07:14 2022] GET /identity/v3/endpoints?service_id=c4e9e15e1f634af38d6ea92c92d5d8cf&interface=public®ion_id=RegionOne => generated 194 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:14.652179 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dd8d0e-e5e5-40a9-b8f2-94b92331e738 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:14.736508 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1ea0da-fdc9-47e5-9e41-2eafaba92f4d None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:14.747959 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:14.750949 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:14.792904 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d3d552-1d59-4f3f-b347-4e57fed78852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:14.794015 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3d552-1d59-4f3f-b347-4e57fed78852 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:14.794507 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3d552-1d59-4f3f-b347-4e57fed78852 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:14.794998 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3d552-1d59-4f3f-b347-4e57fed78852 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:14.795106 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dd8d0e-e5e5-40a9-b8f2-94b92331e738 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:14.796290 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d3d552-1d59-4f3f-b347-4e57fed78852 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:14.796835 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d3d552-1d59-4f3f-b347-4e57fed78852 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:14.799343 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d3d552-1d59-4f3f-b347-4e57fed78852 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:14.823342 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 445/902] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:07:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:14.830479 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d3d552-1d59-4f3f-b347-4e57fed78852 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:14.833427 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c99f9-5c1d-40d7-8961-6ca093abe996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:14.834695 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c99f9-5c1d-40d7-8961-6ca093abe996 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:14.836936 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c99f9-5c1d-40d7-8961-6ca093abe996 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:14.837733 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c99f9-5c1d-40d7-8961-6ca093abe996 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:14.839063 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503c99f9-5c1d-40d7-8961-6ca093abe996 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:14.841560 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503c99f9-5c1d-40d7-8961-6ca093abe996 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:14.865903 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 458/903] 10.222.0.71 () {60 vars in 1141 bytes} [Mon Jul 4 09:07:14 2022] POST /identity/v3/services => generated 241 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:07:14.867584 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c99f9-5c1d-40d7-8961-6ca093abe996 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:14.972184 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d3d552-1d59-4f3f-b347-4e57fed78852 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:15.019506 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174490d7-253d-4d9d-85ae-83768d813497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:15.020345 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174490d7-253d-4d9d-85ae-83768d813497 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:15.020764 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174490d7-253d-4d9d-85ae-83768d813497 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:15.021063 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174490d7-253d-4d9d-85ae-83768d813497 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:15.022168 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174490d7-253d-4d9d-85ae-83768d813497 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:15.023954 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174490d7-253d-4d9d-85ae-83768d813497 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:15.031394 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 446/904] 10.222.0.71 () {56 vars in 1087 bytes} [Mon Jul 4 09:07:14 2022] GET /identity/v3/services => generated 1497 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:07:15.043048 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c99f9-5c1d-40d7-8961-6ca093abe996 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:15.063224 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:15.088962 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9577b1-c7aa-4144-ab0c-8ab27a727aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:15.090090 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9577b1-c7aa-4144-ab0c-8ab27a727aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:15.090553 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9577b1-c7aa-4144-ab0c-8ab27a727aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:15.090840 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9577b1-c7aa-4144-ab0c-8ab27a727aa2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:15.100587 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9577b1-c7aa-4144-ab0c-8ab27a727aa2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:15.102768 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9577b1-c7aa-4144-ab0c-8ab27a727aa2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:15.120489 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-db9577b1-c7aa-4144-ab0c-8ab27a727aa2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 09:07:15.120489 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:15.120489 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:15.120489 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:15.120489 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:15.120489 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:15.120489 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:15.120489 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:15.120489 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:15.120489 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:15.120489 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:15.120489 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:15.120489 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:07:15.120489 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:07:15.120489 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:07:15.120489 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:07:15.121702 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:15.121702 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:15.121702 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:07:15.121702 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:07:15.121702 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:07:15.121702 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:07:15.121702 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:15.121702 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:15.121702 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:15.121702 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:15.121702 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:15.121702 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:15.121702 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:15.121702 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:15.121702 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:15.121702 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:15.121702 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:15.121702 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:15.122719 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:15.122719 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:15.122719 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:07:15.122719 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:07:15.122719 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:07:15.122719 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:07:15.122719 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:07:15.122719 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:07:15.122719 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 09:07:15.122719 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:07:15.122719 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 459/905] 10.222.0.71 () {56 vars in 1117 bytes} [Mon Jul 4 09:07:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:15.127745 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174490d7-253d-4d9d-85ae-83768d813497 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:15.166248 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:15.246943 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174490d7-253d-4d9d-85ae-83768d813497 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:15.284007 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc5acc9-9119-4fbd-8c44-bcb4439464e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:15.284689 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5acc9-9119-4fbd-8c44-bcb4439464e1 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:15.284975 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5acc9-9119-4fbd-8c44-bcb4439464e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:15.285372 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5acc9-9119-4fbd-8c44-bcb4439464e1 None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:15.286228 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc5acc9-9119-4fbd-8c44-bcb4439464e1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:15.286544 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc5acc9-9119-4fbd-8c44-bcb4439464e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:15.287980 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc5acc9-9119-4fbd-8c44-bcb4439464e1 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:15.306183 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 447/906] 10.222.0.71 () {56 vars in 1149 bytes} [Mon Jul 4 09:07:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:15.311669 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9577b1-c7aa-4144-ab0c-8ab27a727aa2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:15.418306 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9577b1-c7aa-4144-ab0c-8ab27a727aa2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:15.458649 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc4847-6b76-4681-8fd3-4a4498f2b6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:15.459300 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc4847-6b76-4681-8fd3-4a4498f2b6aa None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:15.459581 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc4847-6b76-4681-8fd3-4a4498f2b6aa None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:15.459858 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc4847-6b76-4681-8fd3-4a4498f2b6aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:15.510682 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:15.533185 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcc4847-6b76-4681-8fd3-4a4498f2b6aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:15.534644 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcc4847-6b76-4681-8fd3-4a4498f2b6aa None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:15.638731 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 460/907] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:07:15 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:07:15.644459 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc5acc9-9119-4fbd-8c44-bcb4439464e1 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:15.737950 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:15.755126 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc5acc9-9119-4fbd-8c44-bcb4439464e1 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:15.792083 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31f1e1b-3f71-4a55-98ee-1d6807bece17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:15.793292 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31f1e1b-3f71-4a55-98ee-1d6807bece17 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:15.793805 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31f1e1b-3f71-4a55-98ee-1d6807bece17 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:15.794293 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31f1e1b-3f71-4a55-98ee-1d6807bece17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:15.805308 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31f1e1b-3f71-4a55-98ee-1d6807bece17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:15.807046 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31f1e1b-3f71-4a55-98ee-1d6807bece17 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:15.825496 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-b31f1e1b-3f71-4a55-98ee-1d6807bece17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:07:15.825496 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:15.825496 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:15.825496 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:15.825496 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:15.825496 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:15.825496 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:15.825496 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:15.825496 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:15.825496 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:15.825496 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:15.825496 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:15.825496 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:07:15.825496 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:07:15.825496 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:07:15.826433 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:07:15.826433 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:15.826433 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:15.826433 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:15.826433 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:15.826433 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:15.826433 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:15.826433 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:15.826433 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:15.826433 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:15.826433 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:15.826433 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:15.826433 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:15.826433 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:15.826433 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:15.826433 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:15.826433 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:15.827343 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:07:15.827343 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:07:15.827343 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:07:15.827343 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:15.833135 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 448/908] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:07:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:15.838385 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc4847-6b76-4681-8fd3-4a4498f2b6aa None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:15.919264 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:15.941262 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc4847-6b76-4681-8fd3-4a4498f2b6aa None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:15.977638 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0f162e-5283-41b1-ad7b-ab82eca3b983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:15.978306 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0f162e-5283-41b1-ad7b-ab82eca3b983 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:15.978592 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0f162e-5283-41b1-ad7b-ab82eca3b983 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:15.978869 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0f162e-5283-41b1-ad7b-ab82eca3b983 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:15.979652 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0f162e-5283-41b1-ad7b-ab82eca3b983 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:15.979980 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0f162e-5283-41b1-ad7b-ab82eca3b983 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:15.981368 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0f162e-5283-41b1-ad7b-ab82eca3b983 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:15.993476 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 461/909] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:07:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:15.999103 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31f1e1b-3f71-4a55-98ee-1d6807bece17 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:16.108682 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31f1e1b-3f71-4a55-98ee-1d6807bece17 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:16.147522 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898ef2d7-7eea-4323-a7d8-2ebd5582a22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:16.149749 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898ef2d7-7eea-4323-a7d8-2ebd5582a22d None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:16.150269 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898ef2d7-7eea-4323-a7d8-2ebd5582a22d None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:16.150760 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898ef2d7-7eea-4323-a7d8-2ebd5582a22d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:16.158761 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898ef2d7-7eea-4323-a7d8-2ebd5582a22d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:16.160519 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898ef2d7-7eea-4323-a7d8-2ebd5582a22d None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:16.162501 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:16.176003 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-898ef2d7-7eea-4323-a7d8-2ebd5582a22d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:07:16.176003 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:16.176003 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:16.176003 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:16.176003 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:16.176003 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:16.176003 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:16.176003 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:16.176003 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:16.176003 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:16.176003 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:16.176003 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:16.176003 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:07:16.176003 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:07:16.176003 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:07:16.177008 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:07:16.177008 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:16.177008 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:16.177008 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:16.177008 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:16.177008 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:16.177008 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:16.177008 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:16.177008 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:16.177008 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:16.177008 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:16.177008 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:16.177008 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:16.177008 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:16.177008 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:16.177008 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:16.177008 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:16.178021 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:07:16.178021 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:07:16.178021 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:07:16.178021 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:07:16.178021 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:07:16.178021 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:07:16.178021 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:07:16.178021 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:16.178482 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ccdcff-5702-48f7-af41-e43f181ec3b7 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:16.178482 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ccdcff-5702-48f7-af41-e43f181ec3b7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:16.178482 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ccdcff-5702-48f7-af41-e43f181ec3b7 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:16.179330 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 462/910] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:07:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:16.180500 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 449/911] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 4 09:07:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:16.185599 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a1231-b6e7-4352-94c8-6a12a6e5d7f9 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:16.186150 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a1231-b6e7-4352-94c8-6a12a6e5d7f9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:16.186566 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898ef2d7-7eea-4323-a7d8-2ebd5582a22d None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:16.186865 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a1231-b6e7-4352-94c8-6a12a6e5d7f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:16.204747 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-990a1231-b6e7-4352-94c8-6a12a6e5d7f9 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:16.206624 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-990a1231-b6e7-4352-94c8-6a12a6e5d7f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:16.297140 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 463/912] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:07:16.306295 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898ef2d7-7eea-4323-a7d8-2ebd5582a22d None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:16.314162 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990a1231-b6e7-4352-94c8-6a12a6e5d7f9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:16.347806 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c83586-03f4-4c1a-901e-b96ece73dfc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:16.348481 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c83586-03f4-4c1a-901e-b96ece73dfc4 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:16.348893 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c83586-03f4-4c1a-901e-b96ece73dfc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:16.349217 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c83586-03f4-4c1a-901e-b96ece73dfc4 None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:16.350206 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c83586-03f4-4c1a-901e-b96ece73dfc4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:16.350520 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c83586-03f4-4c1a-901e-b96ece73dfc4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:16.352423 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c83586-03f4-4c1a-901e-b96ece73dfc4 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:16.369119 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 450/913] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 4 09:07:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:16.374796 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c83586-03f4-4c1a-901e-b96ece73dfc4 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:16.407037 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:16.441978 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:16.445429 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990a1231-b6e7-4352-94c8-6a12a6e5d7f9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:16.482926 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def04864-ffd8-4526-a9a0-f0c7b79b3339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:16.484043 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def04864-ffd8-4526-a9a0-f0c7b79b3339 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:16.484542 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def04864-ffd8-4526-a9a0-f0c7b79b3339 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:16.485150 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def04864-ffd8-4526-a9a0-f0c7b79b3339 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:16.486019 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def04864-ffd8-4526-a9a0-f0c7b79b3339 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:16.487582 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def04864-ffd8-4526-a9a0-f0c7b79b3339 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:16.489924 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c83586-03f4-4c1a-901e-b96ece73dfc4 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:16.500500 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-def04864-ffd8-4526-a9a0-f0c7b79b3339 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 04 09:07:16.500500 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:16.500500 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:07:16.500500 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:07:16.500500 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:07:16.500500 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:07:16.500500 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:07:16.500500 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:07:16.500500 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 04 09:07:16.500500 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:07:16.500500 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:07:16.500500 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:07:16.500500 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:16.500500 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:16.500500 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:16.500500 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:16.500500 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:16.501597 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:16.501597 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:16.501597 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:16.501597 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:16.501597 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:16.501597 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:16.501597 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:07:16.501597 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:07:16.501597 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:07:16.501597 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:07:16.501597 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:16.501597 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:16.501597 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:16.501597 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:16.501597 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:16.501597 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:16.502476 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:16.502476 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:16.502476 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:16.502476 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:16.502476 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:16.502476 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:16.502476 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:16.502476 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:16.502476 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:16.502476 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:16.502476 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:07:16.502476 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:07:16.502476 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 04 09:07:16.502476 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:07:16.509244 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 464/914] 10.222.0.71 () {56 vars in 1114 bytes} [Mon Jul 4 09:07:16 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:07:16.511881 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def04864-ffd8-4526-a9a0-f0c7b79b3339 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:16.527194 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d0f031-7add-43aa-b4bc-890828ec6e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:16.527857 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d0f031-7add-43aa-b4bc-890828ec6e59 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:16.528186 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d0f031-7add-43aa-b4bc-890828ec6e59 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:16.528472 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d0f031-7add-43aa-b4bc-890828ec6e59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:16.529349 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d0f031-7add-43aa-b4bc-890828ec6e59 None admin] RBAC: Adding query filter params (role.id=86b73ebb032e430c9c08b78a01898cd7, scope.project.id=4076e316243946e980ce47386b18b28f, user.id=78668b939c5847b1b73e9c28b5d8db7f) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:16.529668 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d0f031-7add-43aa-b4bc-890828ec6e59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:16.531162 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d0f031-7add-43aa-b4bc-890828ec6e59 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:16.545238 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 451/915] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 4 09:07:16 2022] GET /identity/v3/role_assignments?user.id=78668b939c5847b1b73e9c28b5d8db7f&scope.project.id=4076e316243946e980ce47386b18b28f&role.id=86b73ebb032e430c9c08b78a01898cd7 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:16.629600 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def04864-ffd8-4526-a9a0-f0c7b79b3339 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:16.665406 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:16.671856 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3562156c-dfc3-4f70-b3fa-a4a666605c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:16.672917 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3562156c-dfc3-4f70-b3fa-a4a666605c56 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:16.673453 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3562156c-dfc3-4f70-b3fa-a4a666605c56 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:16.673952 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3562156c-dfc3-4f70-b3fa-a4a666605c56 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:16.674885 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3562156c-dfc3-4f70-b3fa-a4a666605c56 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:16.675425 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3562156c-dfc3-4f70-b3fa-a4a666605c56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:16.677050 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3562156c-dfc3-4f70-b3fa-a4a666605c56 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:16.683268 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 465/916] 10.222.0.71 () {56 vars in 1114 bytes} [Mon Jul 4 09:07:16 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:16.689335 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d0f031-7add-43aa-b4bc-890828ec6e59 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:16.797007 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d0f031-7add-43aa-b4bc-890828ec6e59 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:16.821481 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:16.835003 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d4115c-258e-4f0a-923e-84bc070f1343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:16.835853 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d4115c-258e-4f0a-923e-84bc070f1343 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:16.836138 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d4115c-258e-4f0a-923e-84bc070f1343 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:16.836419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d4115c-258e-4f0a-923e-84bc070f1343 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:16.837175 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d4115c-258e-4f0a-923e-84bc070f1343 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:16.837742 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d4115c-258e-4f0a-923e-84bc070f1343 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:16.839120 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d4115c-258e-4f0a-923e-84bc070f1343 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:16.844652 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 452/917] 10.222.0.71 () {56 vars in 1114 bytes} [Mon Jul 4 09:07:16 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:16.850329 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3562156c-dfc3-4f70-b3fa-a4a666605c56 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:16.956326 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3562156c-dfc3-4f70-b3fa-a4a666605c56 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:16.992286 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0683fc-84c4-4d3b-9c14-ce8425cca68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:16.993015 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0683fc-84c4-4d3b-9c14-ce8425cca68f None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:16.993348 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0683fc-84c4-4d3b-9c14-ce8425cca68f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:16.993631 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0683fc-84c4-4d3b-9c14-ce8425cca68f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:16.994398 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0683fc-84c4-4d3b-9c14-ce8425cca68f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:16.995772 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0683fc-84c4-4d3b-9c14-ce8425cca68f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:17.017383 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 466/918] 10.222.0.71 () {60 vars in 1144 bytes} [Mon Jul 4 09:07:16 2022] POST /identity/v3/endpoints => generated 346 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:07:17.124238 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9d9778-2329-4a15-a092-f0c0cf49086f None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:17.124571 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9d9778-2329-4a15-a092-f0c0cf49086f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:17.124930 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9d9778-2329-4a15-a092-f0c0cf49086f None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:17.126655 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 453/919] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:07:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:17.133266 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e32fd5-9d3b-48a3-b63b-6607fdb2a45b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:17.133563 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e32fd5-9d3b-48a3-b63b-6607fdb2a45b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:17.133845 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e32fd5-9d3b-48a3-b63b-6607fdb2a45b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:17.145287 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a4535-9cee-4adb-84fc-7bee226b545d None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:17.145660 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a4535-9cee-4adb-84fc-7bee226b545d None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:17.145851 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a4535-9cee-4adb-84fc-7bee226b545d None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:17.148191 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 454/920] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:07:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:17.153408 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dfbeee-4c7f-4472-9742-c9d7836b15a2 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:17.153817 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dfbeee-4c7f-4472-9742-c9d7836b15a2 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:17.154198 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dfbeee-4c7f-4472-9742-c9d7836b15a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:17.155015 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-59e32fd5-9d3b-48a3-b63b-6607fdb2a45b None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:17.156231 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-59e32fd5-9d3b-48a3-b63b-6607fdb2a45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:17.173306 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-00dfbeee-4c7f-4472-9742-c9d7836b15a2 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:17.174560 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-00dfbeee-4c7f-4472-9742-c9d7836b15a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:17.211665 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:17.244446 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 467/921] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:17 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:07:17.250131 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 455/922] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:17 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:07:17.268276 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e32fd5-9d3b-48a3-b63b-6607fdb2a45b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:17.278225 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dfbeee-4c7f-4472-9742-c9d7836b15a2 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:17.396656 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dfbeee-4c7f-4472-9742-c9d7836b15a2 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:17.408267 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e32fd5-9d3b-48a3-b63b-6607fdb2a45b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:17.416804 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:17.436854 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953afddb-b5e4-4e0d-9cc8-48e6b369fed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:17.437903 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953afddb-b5e4-4e0d-9cc8-48e6b369fed2 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:17.438168 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953afddb-b5e4-4e0d-9cc8-48e6b369fed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:17.438448 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953afddb-b5e4-4e0d-9cc8-48e6b369fed2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:17.439100 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953afddb-b5e4-4e0d-9cc8-48e6b369fed2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:17.440585 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953afddb-b5e4-4e0d-9cc8-48e6b369fed2 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:17.447046 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd7b9ec-ab59-4dd4-a168-e75761113542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:17.447857 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd7b9ec-ab59-4dd4-a168-e75761113542 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:17.448142 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd7b9ec-ab59-4dd4-a168-e75761113542 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:17.448423 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd7b9ec-ab59-4dd4-a168-e75761113542 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:17.449103 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd7b9ec-ab59-4dd4-a168-e75761113542 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:17.450473 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd7b9ec-ab59-4dd4-a168-e75761113542 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:17.454852 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-953afddb-b5e4-4e0d-9cc8-48e6b369fed2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 04 09:07:17.454852 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:17.454852 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:07:17.454852 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:07:17.454852 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:07:17.454852 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:07:17.454852 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:07:17.454852 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:07:17.454852 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 04 09:07:17.454852 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:17.454852 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:07:17.454852 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:17.454852 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:17.454852 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:17.454852 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:17.454852 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:17.455776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:17.455776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:17.455776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:17.455776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:17.455776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:17.455776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:17.455776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:17.455776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:07:17.455776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:07:17.455776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:07:17.455776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:07:17.455776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:17.455776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:17.455776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:17.455776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:17.455776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:17.456614 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:17.456614 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:17.456614 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:17.456614 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:17.456614 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:17.456614 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:17.456614 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:17.456614 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:17.456614 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:17.456614 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:17.456614 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:17.456614 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:07:17.456614 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:07:17.456614 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 04 09:07:17.456614 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:17.461182 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 456/923] 10.222.0.71 () {56 vars in 1132 bytes} [Mon Jul 4 09:07:17 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:07:17.462939 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953afddb-b5e4-4e0d-9cc8-48e6b369fed2 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:17.463813 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-ddd7b9ec-ab59-4dd4-a168-e75761113542 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 09:07:17.463813 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:17.463813 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:07:17.463813 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:07:17.463813 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:07:17.463813 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:07:17.463813 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:07:17.463813 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:07:17.463813 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 09:07:17.463813 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:07:17.463813 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:07:17.463813 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:07:17.463813 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:17.463813 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:17.463813 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:17.463813 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:17.463813 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:17.465883 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:17.465883 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:17.465883 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:17.465883 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:17.465883 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:17.465883 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:17.465883 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:07:17.465883 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:07:17.465883 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:07:17.465883 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:07:17.465883 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:17.465883 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:17.465883 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:17.465883 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:17.465883 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:17.465883 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:17.466934 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:17.466934 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:17.466934 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:17.466934 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:17.466934 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:17.466934 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:17.466934 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:17.466934 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:17.466934 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:17.466934 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:17.466934 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:07:17.466934 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:07:17.466934 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 09:07:17.466934 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:07:17.466934 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 468/924] 10.222.0.71 () {56 vars in 1126 bytes} [Mon Jul 4 09:07:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:17.474018 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd7b9ec-ab59-4dd4-a168-e75761113542 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:17.530647 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:17.583279 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953afddb-b5e4-4e0d-9cc8-48e6b369fed2 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:17.605004 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd7b9ec-ab59-4dd4-a168-e75761113542 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:17.622529 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b31f4be-14cf-4a83-aa0a-fde51e1ebead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:17.623328 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b31f4be-14cf-4a83-aa0a-fde51e1ebead None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:17.623713 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b31f4be-14cf-4a83-aa0a-fde51e1ebead None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:17.624088 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b31f4be-14cf-4a83-aa0a-fde51e1ebead None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:17.624896 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b31f4be-14cf-4a83-aa0a-fde51e1ebead None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:17.625324 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b31f4be-14cf-4a83-aa0a-fde51e1ebead None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:17.626802 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b31f4be-14cf-4a83-aa0a-fde51e1ebead None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:17.635570 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 457/925] 10.222.0.71 () {56 vars in 1126 bytes} [Mon Jul 4 09:07:17 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:07:17.640787 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b31f4be-14cf-4a83-aa0a-fde51e1ebead None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:17.648457 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65d9208-c7f6-4ef4-a2bb-d1da4225b051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:17.649438 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d9208-c7f6-4ef4-a2bb-d1da4225b051 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:17.649951 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d9208-c7f6-4ef4-a2bb-d1da4225b051 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:17.650441 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d9208-c7f6-4ef4-a2bb-d1da4225b051 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:17.651503 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65d9208-c7f6-4ef4-a2bb-d1da4225b051 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:17.652028 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65d9208-c7f6-4ef4-a2bb-d1da4225b051 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:17.653733 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65d9208-c7f6-4ef4-a2bb-d1da4225b051 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:17.659234 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 469/926] 10.222.0.71 () {56 vars in 1122 bytes} [Mon Jul 4 09:07:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:17.664969 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65d9208-c7f6-4ef4-a2bb-d1da4225b051 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:17.699230 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:17.760337 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b31f4be-14cf-4a83-aa0a-fde51e1ebead None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:17.785254 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65d9208-c7f6-4ef4-a2bb-d1da4225b051 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:17.793564 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:17.797736 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda70acc-735a-4b03-ba3d-246297c3405d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:17.798435 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda70acc-735a-4b03-ba3d-246297c3405d None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:17.798701 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda70acc-735a-4b03-ba3d-246297c3405d None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:17.798986 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda70acc-735a-4b03-ba3d-246297c3405d None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:17.799703 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda70acc-735a-4b03-ba3d-246297c3405d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:17.800067 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda70acc-735a-4b03-ba3d-246297c3405d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:17.801418 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda70acc-735a-4b03-ba3d-246297c3405d None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:17.808605 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 458/927] 10.222.0.71 () {56 vars in 1126 bytes} [Mon Jul 4 09:07:17 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:17.813734 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda70acc-735a-4b03-ba3d-246297c3405d None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:17.829413 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b68a942-58ee-49f0-98da-bf174971bd98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:17.830102 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b68a942-58ee-49f0-98da-bf174971bd98 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:17.830444 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b68a942-58ee-49f0-98da-bf174971bd98 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:17.830727 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b68a942-58ee-49f0-98da-bf174971bd98 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:17.831445 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b68a942-58ee-49f0-98da-bf174971bd98 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:17.831756 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b68a942-58ee-49f0-98da-bf174971bd98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:17.834003 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b68a942-58ee-49f0-98da-bf174971bd98 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:17.847224 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 470/928] 10.222.0.71 () {56 vars in 1122 bytes} [Mon Jul 4 09:07:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:07:17.848262 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b68a942-58ee-49f0-98da-bf174971bd98 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:17.941714 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda70acc-735a-4b03-ba3d-246297c3405d None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:17.953087 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:17.978168 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b68a942-58ee-49f0-98da-bf174971bd98 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:17.986583 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c4a7c1-ec54-4d95-ba66-ab648a97e519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:17.987453 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c4a7c1-ec54-4d95-ba66-ab648a97e519 None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:17.987895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c4a7c1-ec54-4d95-ba66-ab648a97e519 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:17.988771 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c4a7c1-ec54-4d95-ba66-ab648a97e519 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:17.989697 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c4a7c1-ec54-4d95-ba66-ab648a97e519 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:17.992448 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c4a7c1-ec54-4d95-ba66-ab648a97e519 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:18.025854 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 459/929] 10.222.0.71 () {60 vars in 1144 bytes} [Mon Jul 4 09:07:17 2022] POST /identity/v3/endpoints => generated 347 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:07:18.031929 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848888b5-8489-4b2f-8ba2-145daa2e52d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:18.032784 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848888b5-8489-4b2f-8ba2-145daa2e52d0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:18.033228 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848888b5-8489-4b2f-8ba2-145daa2e52d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:18.033614 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848888b5-8489-4b2f-8ba2-145daa2e52d0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:18.034636 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848888b5-8489-4b2f-8ba2-145daa2e52d0 None admin] RBAC: Adding query filter params (interface=public, service_id=fd599f217ead4e98aeac1954f00c098b, region_id=RegionOne) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:18.035166 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848888b5-8489-4b2f-8ba2-145daa2e52d0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:18.036726 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848888b5-8489-4b2f-8ba2-145daa2e52d0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:18.044450 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 471/930] 10.222.0.71 () {56 vars in 1251 bytes} [Mon Jul 4 09:07:17 2022] GET /identity/v3/endpoints?service_id=fd599f217ead4e98aeac1954f00c098b&interface=public®ion_id=RegionOne => generated 194 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:18.050489 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c4a7c1-ec54-4d95-ba66-ab648a97e519 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:18.172768 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c4a7c1-ec54-4d95-ba66-ab648a97e519 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:18.209496 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081a4636-d8dd-4438-939c-baea0bc9a3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:18.210774 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a4636-d8dd-4438-939c-baea0bc9a3ff None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:18.211064 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a4636-d8dd-4438-939c-baea0bc9a3ff None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:18.211344 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a4636-d8dd-4438-939c-baea0bc9a3ff None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:18.212034 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081a4636-d8dd-4438-939c-baea0bc9a3ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:18.213698 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081a4636-d8dd-4438-939c-baea0bc9a3ff None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:18.220063 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 460/931] 10.222.0.71 () {56 vars in 1087 bytes} [Mon Jul 4 09:07:18 2022] GET /identity/v3/services => generated 1497 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:07:18.679720 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e565fb0-2995-4dc8-aa1f-c0d52d50e031 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:18.680322 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e565fb0-2995-4dc8-aa1f-c0d52d50e031 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:18.680322 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e565fb0-2995-4dc8-aa1f-c0d52d50e031 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:18.681983 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 472/932] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:07:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:18.687222 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac13b144-ae94-4ff8-95c1-bd8d4d794013 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:18.687623 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac13b144-ae94-4ff8-95c1-bd8d4d794013 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:18.687790 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac13b144-ae94-4ff8-95c1-bd8d4d794013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:18.706003 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-ac13b144-ae94-4ff8-95c1-bd8d4d794013 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:18.707207 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ac13b144-ae94-4ff8-95c1-bd8d4d794013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:18.732947 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:18.780908 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 461/933] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:18 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:07:18.804481 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e565fb0-2995-4dc8-aa1f-c0d52d50e031 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:18.836916 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:18.911459 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e565fb0-2995-4dc8-aa1f-c0d52d50e031 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:18.947904 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bae56b-06a1-464d-aca0-9c91ce880dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:18.949067 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bae56b-06a1-464d-aca0-9c91ce880dd5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:18.949553 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bae56b-06a1-464d-aca0-9c91ce880dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:18.950051 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bae56b-06a1-464d-aca0-9c91ce880dd5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:18.961374 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bae56b-06a1-464d-aca0-9c91ce880dd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:07:18.970432 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-83bae56b-06a1-464d-aca0-9c91ce880dd5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 09:07:18.970432 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:18.970432 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:18.970432 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:18.970432 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:18.970432 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:18.970432 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:18.970432 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:18.970432 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:18.970432 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:18.970432 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:18.970432 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:18.970432 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:07:18.970432 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:07:18.970432 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:07:18.970432 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:07:18.971421 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:07:18.971421 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:07:18.971421 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:07:18.971421 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:07:18.971421 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:18.971421 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:18.971421 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:18.971421 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:18.971421 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:18.971421 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:18.971421 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:18.971421 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:18.971421 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:18.971421 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:18.971421 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:18.971421 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:18.972264 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:18.972264 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:18.972264 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:18.972264 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:18.972264 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:07:18.972264 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:07:18.972264 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:07:18.972264 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:07:18.972264 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:07:18.972264 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:07:18.972264 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 09:07:18.972264 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:07:18.974999 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 473/934] 10.222.0.71 () {56 vars in 1120 bytes} [Mon Jul 4 09:07:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:18.984795 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac13b144-ae94-4ff8-95c1-bd8d4d794013 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:19.085117 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac13b144-ae94-4ff8-95c1-bd8d4d794013 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:19.122529 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a54431-720e-45ff-9dab-7006ff17d127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:19.123555 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a54431-720e-45ff-9dab-7006ff17d127 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:19.124037 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a54431-720e-45ff-9dab-7006ff17d127 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:19.124531 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a54431-720e-45ff-9dab-7006ff17d127 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:19.125500 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a54431-720e-45ff-9dab-7006ff17d127 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:19.126013 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a54431-720e-45ff-9dab-7006ff17d127 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:19.127691 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a54431-720e-45ff-9dab-7006ff17d127 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:19.136711 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 462/935] 10.222.0.71 () {56 vars in 1115 bytes} [Mon Jul 4 09:07:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:19.142397 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bae56b-06a1-464d-aca0-9c91ce880dd5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:19.245423 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:19.250678 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d6a25-a365-449f-afaa-55397dd149b9 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:19.250964 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d6a25-a365-449f-afaa-55397dd149b9 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:19.251237 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d6a25-a365-449f-afaa-55397dd149b9 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:19.251929 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bae56b-06a1-464d-aca0-9c91ce880dd5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:19.252731 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 463/936] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:07:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:19.261757 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e645f-1ae7-4681-8dc3-82f9f7cb1331 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:19.262127 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e645f-1ae7-4681-8dc3-82f9f7cb1331 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:19.262481 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e645f-1ae7-4681-8dc3-82f9f7cb1331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:19.279926 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-2a2e645f-1ae7-4681-8dc3-82f9f7cb1331 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:19.281145 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-2a2e645f-1ae7-4681-8dc3-82f9f7cb1331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:19.295875 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:19.296729 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17107040-938a-4e27-9096-0a7e6d52fc79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:19.297584 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17107040-938a-4e27-9096-0a7e6d52fc79 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:19.297871 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17107040-938a-4e27-9096-0a7e6d52fc79 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:19.298162 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17107040-938a-4e27-9096-0a7e6d52fc79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:19.312258 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17107040-938a-4e27-9096-0a7e6d52fc79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:19.313849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17107040-938a-4e27-9096-0a7e6d52fc79 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:19.333534 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-17107040-938a-4e27-9096-0a7e6d52fc79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:07:19.333534 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:19.333534 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:19.333534 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:19.333534 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:19.333534 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:19.333534 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:19.333534 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:19.333534 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:19.333534 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:19.333534 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:19.333534 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:19.333534 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:07:19.333534 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:07:19.333534 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:07:19.334692 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:07:19.334692 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:19.334692 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:19.334692 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:19.334692 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:19.334692 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:19.334692 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:19.334692 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:19.334692 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:19.334692 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:19.334692 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:19.334692 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:19.334692 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:19.334692 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:19.334692 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:19.334692 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:19.334692 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:19.336207 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:07:19.336207 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:07:19.336207 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:07:19.336207 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:07:19.336207 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 474/937] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:07:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:19.341057 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17107040-938a-4e27-9096-0a7e6d52fc79 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:19.353535 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 464/938] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:07:19.378004 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2e645f-1ae7-4681-8dc3-82f9f7cb1331 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:19.458260 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17107040-938a-4e27-9096-0a7e6d52fc79 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:19.496902 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf81c6e-1aa4-4087-a9e5-50cb7bc69db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:19.497819 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf81c6e-1aa4-4087-a9e5-50cb7bc69db9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:19.498271 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf81c6e-1aa4-4087-a9e5-50cb7bc69db9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:19.498601 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2e645f-1ae7-4681-8dc3-82f9f7cb1331 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:19.498881 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf81c6e-1aa4-4087-a9e5-50cb7bc69db9 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:19.499742 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf81c6e-1aa4-4087-a9e5-50cb7bc69db9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:19.500286 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf81c6e-1aa4-4087-a9e5-50cb7bc69db9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:19.501984 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf81c6e-1aa4-4087-a9e5-50cb7bc69db9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:19.520195 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 475/939] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:07:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:19.521858 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf81c6e-1aa4-4087-a9e5-50cb7bc69db9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:19.524808 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:19.535420 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4709465-258e-4aaf-91ee-29145f647cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:19.536097 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4709465-258e-4aaf-91ee-29145f647cf1 None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:19.536384 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4709465-258e-4aaf-91ee-29145f647cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:19.536664 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4709465-258e-4aaf-91ee-29145f647cf1 None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:19.537453 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4709465-258e-4aaf-91ee-29145f647cf1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:19.539015 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4709465-258e-4aaf-91ee-29145f647cf1 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:19.540381 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: WARNING keystone.server.flask.common [None req-f4709465-258e-4aaf-91ee-29145f647cf1 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 04 09:07:19.558979 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.sql.core [None req-f4709465-258e-4aaf-91ee-29145f647cf1 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 04 09:07:19.558979 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 04 09:07:19.558979 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [parameters: {'id': '67fad3bce6574cc1991d8469dc5c81f6', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 04 09:07:19.558979 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80171) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 04 09:07:19.580582 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-f4709465-258e-4aaf-91ee-29145f647cf1 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 67fad3bce6574cc1991d8469dc5c81f6.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 67fad3bce6574cc1991d8469dc5c81f6. Jul 04 09:07:19.580582 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:19.580582 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 04 09:07:19.580582 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 04 09:07:19.580582 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 04 09:07:19.580582 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 04 09:07:19.580582 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 04 09:07:19.580582 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application result = self._query(query) Jul 04 09:07:19.580582 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 04 09:07:19.580582 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application conn.query(q) Jul 04 09:07:19.580582 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 04 09:07:19.580582 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 04 09:07:19.580582 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 04 09:07:19.581712 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application result.read() Jul 04 09:07:19.581712 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 04 09:07:19.581712 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 04 09:07:19.581712 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 04 09:07:19.581712 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 04 09:07:19.581712 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 04 09:07:19.581712 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 04 09:07:19.581712 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 04 09:07:19.581712 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 04 09:07:19.581712 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 04 09:07:19.581712 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:19.581712 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 04 09:07:19.581712 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:19.581712 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:19.581712 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 04 09:07:19.581712 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 04 09:07:19.581712 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 04 09:07:19.581712 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 04 09:07:19.582922 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 04 09:07:19.582922 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application next(self.gen) Jul 04 09:07:19.582922 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 04 09:07:19.582922 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application yield resource Jul 04 09:07:19.582922 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 04 09:07:19.582922 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application next(self.gen) Jul 04 09:07:19.582922 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 04 09:07:19.582922 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self.session.rollback() Jul 04 09:07:19.582922 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 04 09:07:19.582922 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self.force_reraise() Jul 04 09:07:19.582922 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 04 09:07:19.582922 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise self.value Jul 04 09:07:19.582922 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 04 09:07:19.582922 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 04 09:07:19.582922 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 04 09:07:19.582922 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application session.commit() Jul 04 09:07:19.582922 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 04 09:07:19.584050 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 04 09:07:19.584050 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 04 09:07:19.584050 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._prepare_impl() Jul 04 09:07:19.584050 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 04 09:07:19.584050 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self.session.flush() Jul 04 09:07:19.584050 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 04 09:07:19.584050 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._flush(objects) Jul 04 09:07:19.584050 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 04 09:07:19.584050 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 04 09:07:19.584050 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 04 09:07:19.584050 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application compat.raise_( Jul 04 09:07:19.584050 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 04 09:07:19.584050 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception Jul 04 09:07:19.584050 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 04 09:07:19.584050 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application flush_context.execute() Jul 04 09:07:19.584050 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 04 09:07:19.584050 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rec.execute(self) Jul 04 09:07:19.584984 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 04 09:07:19.584984 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 04 09:07:19.584984 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 04 09:07:19.584984 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 04 09:07:19.584984 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 04 09:07:19.584984 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 04 09:07:19.584984 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 04 09:07:19.584984 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 04 09:07:19.584984 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 04 09:07:19.584984 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 04 09:07:19.584984 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 04 09:07:19.584984 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 04 09:07:19.584984 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 04 09:07:19.584984 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 04 09:07:19.584984 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 04 09:07:19.586209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 04 09:07:19.586209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 04 09:07:19.586209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception Jul 04 09:07:19.586209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 04 09:07:19.586209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 04 09:07:19.586209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 04 09:07:19.586209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 04 09:07:19.586209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 04 09:07:19.586209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application result = self._query(query) Jul 04 09:07:19.586209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 04 09:07:19.586209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application conn.query(q) Jul 04 09:07:19.586209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 04 09:07:19.586209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 04 09:07:19.586209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 04 09:07:19.586209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application result.read() Jul 04 09:07:19.586209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 04 09:07:19.586209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 04 09:07:19.587653 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 04 09:07:19.587653 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 04 09:07:19.587653 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 04 09:07:19.587653 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 04 09:07:19.587653 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 04 09:07:19.587653 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 04 09:07:19.587653 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 04 09:07:19.587653 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 04 09:07:19.587653 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [parameters: {'id': '67fad3bce6574cc1991d8469dc5c81f6', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 04 09:07:19.587653 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 04 09:07:19.587653 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:19.587653 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:07:19.587653 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:19.587653 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:19.587653 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 04 09:07:19.588530 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 04 09:07:19.588530 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 04 09:07:19.588530 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 04 09:07:19.588530 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 04 09:07:19.588530 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:19.588530 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:07:19.588530 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:19.588530 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:19.588530 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:19.588530 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:19.588530 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:19.588530 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:19.588530 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:19.588530 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:19.588530 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:19.588530 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:19.588530 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:19.589590 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:19.589590 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 04 09:07:19.589590 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 04 09:07:19.589590 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:19.589590 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:19.589590 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 04 09:07:19.589590 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 04 09:07:19.589590 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 67fad3bce6574cc1991d8469dc5c81f6. Jul 04 09:07:19.589590 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:19.589590 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 465/940] 10.222.0.71 () {60 vars in 1140 bytes} [Mon Jul 4 09:07:19 2022] POST /identity/v3/projects => generated 262 bytes in 205 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:07:19.589590 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4709465-258e-4aaf-91ee-29145f647cf1 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:19.602444 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:19.648825 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf81c6e-1aa4-4087-a9e5-50cb7bc69db9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:19.691895 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ee50a7-4582-4705-86ab-df33a8c0f8bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:19.692771 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee50a7-4582-4705-86ab-df33a8c0f8bb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:19.693098 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee50a7-4582-4705-86ab-df33a8c0f8bb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:19.693388 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee50a7-4582-4705-86ab-df33a8c0f8bb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:19.708615 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ee50a7-4582-4705-86ab-df33a8c0f8bb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:19.712116 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ee50a7-4582-4705-86ab-df33a8c0f8bb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:19.719326 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4709465-258e-4aaf-91ee-29145f647cf1 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:19.735765 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-81ee50a7-4582-4705-86ab-df33a8c0f8bb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 09:07:19.735765 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:19.735765 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:19.735765 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:19.735765 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:19.735765 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:19.735765 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:19.735765 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:19.735765 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:19.735765 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:19.735765 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:19.735765 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:19.735765 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:07:19.735765 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:07:19.735765 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:07:19.735765 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:07:19.736788 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:19.736788 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:19.736788 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:07:19.736788 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:07:19.736788 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:07:19.736788 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:07:19.736788 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:19.736788 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:19.736788 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:19.736788 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:19.736788 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:19.736788 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:19.736788 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:19.736788 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:19.736788 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:19.736788 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:19.736788 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:19.736788 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:19.737873 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:19.737873 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:19.737873 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:07:19.737873 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:07:19.737873 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:07:19.737873 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:07:19.737873 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:07:19.737873 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:07:19.737873 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 09:07:19.737873 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:07:19.741177 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 476/941] 10.222.0.71 () {56 vars in 1117 bytes} [Mon Jul 4 09:07:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:19.743678 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ee50a7-4582-4705-86ab-df33a8c0f8bb None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:19.765734 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072102e7-e589-4372-b4b5-ce870eed1021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:19.766436 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072102e7-e589-4372-b4b5-ce870eed1021 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:19.766725 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072102e7-e589-4372-b4b5-ce870eed1021 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:19.767003 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072102e7-e589-4372-b4b5-ce870eed1021 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:19.774634 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072102e7-e589-4372-b4b5-ce870eed1021 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:19.776526 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072102e7-e589-4372-b4b5-ce870eed1021 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:19.793662 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-072102e7-e589-4372-b4b5-ce870eed1021 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:07:19.793662 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:19.793662 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:19.793662 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:19.793662 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:19.793662 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:19.793662 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:19.793662 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:19.793662 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:19.793662 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:19.793662 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:19.793662 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:19.793662 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:07:19.793662 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:07:19.793662 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:07:19.794784 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:07:19.794784 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:19.794784 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:19.794784 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:19.794784 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:19.794784 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:19.794784 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:19.794784 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:19.794784 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:19.794784 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:19.794784 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:19.794784 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:19.794784 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:19.794784 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:19.794784 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:19.794784 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:19.794784 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:19.796298 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:07:19.796298 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:07:19.796298 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:07:19.796298 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:07:19.796298 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:07:19.796298 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:07:19.796298 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:07:19.796298 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:19.796298 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 466/942] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 4 09:07:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:19.804892 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072102e7-e589-4372-b4b5-ce870eed1021 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:19.831297 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:19.883618 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ee50a7-4582-4705-86ab-df33a8c0f8bb None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:19.887505 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:19.925326 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072102e7-e589-4372-b4b5-ce870eed1021 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:19.932110 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6cdb84-ad35-4152-9741-fb4bd73b5569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:19.933130 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6cdb84-ad35-4152-9741-fb4bd73b5569 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:19.933634 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6cdb84-ad35-4152-9741-fb4bd73b5569 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:19.934111 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6cdb84-ad35-4152-9741-fb4bd73b5569 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:19.935175 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6cdb84-ad35-4152-9741-fb4bd73b5569 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:19.935692 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6cdb84-ad35-4152-9741-fb4bd73b5569 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:19.937350 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6cdb84-ad35-4152-9741-fb4bd73b5569 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:19.956069 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 477/943] 10.222.0.71 () {56 vars in 1149 bytes} [Mon Jul 4 09:07:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:19.961914 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6cdb84-ad35-4152-9741-fb4bd73b5569 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:19.972260 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c315877-0a5f-4ab6-bc0c-02ab1389ca4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:19.973261 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c315877-0a5f-4ab6-bc0c-02ab1389ca4a None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:19.973809 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c315877-0a5f-4ab6-bc0c-02ab1389ca4a None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:19.974290 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c315877-0a5f-4ab6-bc0c-02ab1389ca4a None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:19.975278 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c315877-0a5f-4ab6-bc0c-02ab1389ca4a None admin] RBAC: Adding query filter params (name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:19.976339 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c315877-0a5f-4ab6-bc0c-02ab1389ca4a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:19.978088 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c315877-0a5f-4ab6-bc0c-02ab1389ca4a None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:20.000623 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 467/944] 10.222.0.71 () {56 vars in 1112 bytes} [Mon Jul 4 09:07:19 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:20.091533 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6cdb84-ad35-4152-9741-fb4bd73b5569 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:20.115357 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:20.134300 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e82e20-52a3-4bbc-97a6-f56f22621d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:20.135119 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e82e20-52a3-4bbc-97a6-f56f22621d38 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:20.135403 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e82e20-52a3-4bbc-97a6-f56f22621d38 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:20.135678 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e82e20-52a3-4bbc-97a6-f56f22621d38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:20.147422 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e82e20-52a3-4bbc-97a6-f56f22621d38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:20.149254 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e82e20-52a3-4bbc-97a6-f56f22621d38 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:20.169907 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-d3e82e20-52a3-4bbc-97a6-f56f22621d38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:07:20.169907 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:20.169907 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:20.169907 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:20.169907 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:20.169907 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:20.169907 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:20.169907 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:20.169907 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:20.169907 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:20.169907 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:20.169907 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:20.169907 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:07:20.169907 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:07:20.169907 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:07:20.170907 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:07:20.170907 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:20.170907 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:20.170907 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:20.170907 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:20.170907 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:20.170907 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:20.170907 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:20.170907 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:20.170907 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:20.170907 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:20.170907 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:20.170907 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:20.170907 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:20.170907 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:20.170907 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:20.170907 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:20.171813 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:07:20.171813 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:07:20.171813 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:07:20.171813 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:07:20.171813 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 478/945] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:07:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:07:20.182264 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c315877-0a5f-4ab6-bc0c-02ab1389ca4a None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:20.270690 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d006a40-1c37-42da-b1e6-f166dca68c0b None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:20.271351 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d006a40-1c37-42da-b1e6-f166dca68c0b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:20.271838 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d006a40-1c37-42da-b1e6-f166dca68c0b None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:20.274489 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 479/946] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:07:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:20.279986 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aeafd1-12f0-4e06-85a7-2ccd20a51bfb None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:20.280391 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aeafd1-12f0-4e06-85a7-2ccd20a51bfb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:20.280775 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aeafd1-12f0-4e06-85a7-2ccd20a51bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:20.297116 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c315877-0a5f-4ab6-bc0c-02ab1389ca4a None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:20.300237 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a1aeafd1-12f0-4e06-85a7-2ccd20a51bfb None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:20.301572 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a1aeafd1-12f0-4e06-85a7-2ccd20a51bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:20.327720 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:20.340451 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec537fd-03ae-4e94-ab89-d27d768545f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:20.341153 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec537fd-03ae-4e94-ab89-d27d768545f6 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:20.341449 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec537fd-03ae-4e94-ab89-d27d768545f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:20.341785 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec537fd-03ae-4e94-ab89-d27d768545f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:20.342464 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec537fd-03ae-4e94-ab89-d27d768545f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:20.342773 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec537fd-03ae-4e94-ab89-d27d768545f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:20.344153 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec537fd-03ae-4e94-ab89-d27d768545f6 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:20.361325 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 468/947] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:07:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:20.363099 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec537fd-03ae-4e94-ab89-d27d768545f6 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:20.405387 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 480/948] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:20 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:07:20.405387 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec8008a-7c66-4f08-9002-eae28403bcbb None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:20.405387 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec8008a-7c66-4f08-9002-eae28403bcbb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:20.405387 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec8008a-7c66-4f08-9002-eae28403bcbb None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:20.405387 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 481/949] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:07:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:20.408939 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be91b4-bf7c-4936-b20d-ba08dabbb9b1 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:20.409278 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be91b4-bf7c-4936-b20d-ba08dabbb9b1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:20.409566 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be91b4-bf7c-4936-b20d-ba08dabbb9b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:20.432947 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-66be91b4-bf7c-4936-b20d-ba08dabbb9b1 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:20.434970 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-66be91b4-bf7c-4936-b20d-ba08dabbb9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:20.485640 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec537fd-03ae-4e94-ab89-d27d768545f6 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:20.516480 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 482/950] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:20 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:07:20.517369 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be91b4-bf7c-4936-b20d-ba08dabbb9b1 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:20.531854 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dbe136-7b92-4f79-8703-9c50bc84b153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:20.532989 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dbe136-7b92-4f79-8703-9c50bc84b153 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:20.533473 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dbe136-7b92-4f79-8703-9c50bc84b153 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:20.533901 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dbe136-7b92-4f79-8703-9c50bc84b153 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:20.542438 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dbe136-7b92-4f79-8703-9c50bc84b153 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:20.544483 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dbe136-7b92-4f79-8703-9c50bc84b153 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:20.557197 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:20.559966 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-15dbe136-7b92-4f79-8703-9c50bc84b153 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:07:20.559966 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:20.559966 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:20.559966 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:20.559966 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:20.559966 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:20.559966 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:20.559966 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:20.559966 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:20.559966 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:20.559966 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:20.559966 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:20.559966 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:07:20.559966 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:07:20.559966 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:07:20.560875 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:07:20.560875 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:20.560875 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:20.560875 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:20.560875 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:20.560875 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:20.560875 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:20.560875 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:20.560875 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:20.560875 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:20.560875 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:20.560875 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:20.560875 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:20.560875 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:20.560875 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:20.560875 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:20.560875 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:20.561836 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:07:20.561836 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:07:20.561836 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:07:20.561836 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:07:20.561836 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:07:20.561836 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:07:20.561836 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:07:20.561836 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:20.566050 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 469/951] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 4 09:07:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:20.566902 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dbe136-7b92-4f79-8703-9c50bc84b153 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:20.615694 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:20.637678 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be91b4-bf7c-4936-b20d-ba08dabbb9b1 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:20.672318 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dbe136-7b92-4f79-8703-9c50bc84b153 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:20.673881 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff82234-13f0-4f2d-b3c4-15d253c9e31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:20.674684 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff82234-13f0-4f2d-b3c4-15d253c9e31f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:20.674966 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff82234-13f0-4f2d-b3c4-15d253c9e31f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:20.675244 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff82234-13f0-4f2d-b3c4-15d253c9e31f None admin] PATH_INFO: `/v3/services/compute` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:20.675892 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff82234-13f0-4f2d-b3c4-15d253c9e31f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:20.677288 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff82234-13f0-4f2d-b3c4-15d253c9e31f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:20.689981 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-0ff82234-13f0-4f2d-b3c4-15d253c9e31f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 04 09:07:20.689981 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:20.689981 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:07:20.689981 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:07:20.689981 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:07:20.689981 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:07:20.689981 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:07:20.689981 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:07:20.689981 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 04 09:07:20.689981 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:07:20.689981 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:07:20.689981 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:07:20.689981 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:20.689981 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:20.689981 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:20.689981 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:20.689981 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:20.691085 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:20.691085 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:20.691085 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:20.691085 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:20.691085 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:20.691085 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:20.691085 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:07:20.691085 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:07:20.691085 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:07:20.691085 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:07:20.691085 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:20.691085 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:20.691085 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:20.691085 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:20.691085 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:20.691085 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:20.691956 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:20.691956 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:20.691956 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:20.691956 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:20.691956 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:20.691956 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:20.691956 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:20.691956 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:20.691956 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:20.691956 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:20.691956 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:07:20.691956 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:07:20.691956 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 04 09:07:20.691956 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:07:20.691956 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 483/952] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 4 09:07:20 2022] GET /identity/v3/services/compute => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:20.693365 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff82234-13f0-4f2d-b3c4-15d253c9e31f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:20.715331 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fbb124-5942-41fb-be71-177ff7cb29ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:20.716149 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fbb124-5942-41fb-be71-177ff7cb29ff None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:20.716434 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fbb124-5942-41fb-be71-177ff7cb29ff None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:20.716721 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fbb124-5942-41fb-be71-177ff7cb29ff None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:20.717404 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fbb124-5942-41fb-be71-177ff7cb29ff None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:20.718780 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fbb124-5942-41fb-be71-177ff7cb29ff None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:20.731355 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-61fbb124-5942-41fb-be71-177ff7cb29ff None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 09:07:20.731355 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:20.731355 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:07:20.731355 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:07:20.731355 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:07:20.731355 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:07:20.731355 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:07:20.731355 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:07:20.731355 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 09:07:20.731355 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:20.731355 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:07:20.731355 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:20.731355 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:20.731355 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:20.731355 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:20.731355 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:20.731355 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:20.737794 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:20.737794 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:20.737794 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:20.737794 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:20.737794 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:20.737794 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:20.737794 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:07:20.737794 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:07:20.737794 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:07:20.737794 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:07:20.737794 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:20.737794 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:20.737794 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:20.737794 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:20.737794 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:20.737794 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:20.738940 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:20.738940 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:20.738940 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:20.738940 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:20.738940 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:20.738940 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:20.738940 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:20.738940 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:20.738940 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:20.738940 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:20.738940 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:07:20.738940 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:07:20.738940 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 09:07:20.738940 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:20.738940 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 470/953] 10.222.0.71 () {56 vars in 1126 bytes} [Mon Jul 4 09:07:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:20.739965 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fbb124-5942-41fb-be71-177ff7cb29ff None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:20.799615 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:20.806981 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff82234-13f0-4f2d-b3c4-15d253c9e31f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:20.843929 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0806b3f-d124-426f-933f-a44ec464ed4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:20.844597 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0806b3f-d124-426f-933f-a44ec464ed4b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:20.844874 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0806b3f-d124-426f-933f-a44ec464ed4b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:20.845177 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0806b3f-d124-426f-933f-a44ec464ed4b None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:20.845960 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0806b3f-d124-426f-933f-a44ec464ed4b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:20.846265 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0806b3f-d124-426f-933f-a44ec464ed4b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:20.847685 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0806b3f-d124-426f-933f-a44ec464ed4b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:20.864894 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fbb124-5942-41fb-be71-177ff7cb29ff None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:20.868328 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 484/954] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 4 09:07:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:20.869135 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0806b3f-d124-426f-933f-a44ec464ed4b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:20.876752 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:20.903745 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13020a1f-d9cd-4dc8-ade8-e6c7f1a51408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:20.904427 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13020a1f-d9cd-4dc8-ade8-e6c7f1a51408 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:20.904713 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13020a1f-d9cd-4dc8-ade8-e6c7f1a51408 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:20.905000 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13020a1f-d9cd-4dc8-ade8-e6c7f1a51408 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:20.905744 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13020a1f-d9cd-4dc8-ade8-e6c7f1a51408 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:20.906056 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13020a1f-d9cd-4dc8-ade8-e6c7f1a51408 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:20.907429 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13020a1f-d9cd-4dc8-ade8-e6c7f1a51408 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:20.912791 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 471/955] 10.222.0.71 () {56 vars in 1112 bytes} [Mon Jul 4 09:07:20 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:20.914429 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13020a1f-d9cd-4dc8-ade8-e6c7f1a51408 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:20.982480 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0806b3f-d124-426f-933f-a44ec464ed4b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:21.018042 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8c8f94-a3f4-4e69-9996-324f73f60e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:21.018960 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8c8f94-a3f4-4e69-9996-324f73f60e70 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:21.019400 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8c8f94-a3f4-4e69-9996-324f73f60e70 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:21.019823 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8c8f94-a3f4-4e69-9996-324f73f60e70 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:21.020663 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8c8f94-a3f4-4e69-9996-324f73f60e70 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:21.021158 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8c8f94-a3f4-4e69-9996-324f73f60e70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:21.021271 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13020a1f-d9cd-4dc8-ade8-e6c7f1a51408 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:21.023268 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8c8f94-a3f4-4e69-9996-324f73f60e70 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:21.028073 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 485/956] 10.222.0.71 () {56 vars in 1122 bytes} [Mon Jul 4 09:07:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:21.028838 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8c8f94-a3f4-4e69-9996-324f73f60e70 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:21.052062 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:21.063031 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b6494b-8eea-48d4-86fe-9af58c2c2d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:21.063927 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b6494b-8eea-48d4-86fe-9af58c2c2d79 None admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:21.064216 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b6494b-8eea-48d4-86fe-9af58c2c2d79 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:21.064498 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b6494b-8eea-48d4-86fe-9af58c2c2d79 None admin] PATH_INFO: `/v3/projects/4076e316243946e980ce47386b18b28f/users/78668b939c5847b1b73e9c28b5d8db7f/roles/86b73ebb032e430c9c08b78a01898cd7` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:21.067048 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b6494b-8eea-48d4-86fe-9af58c2c2d79 None admin] RBAC: Authorizing `identity:create_grant(project_id=4076e316243946e980ce47386b18b28f, user_id=78668b939c5847b1b73e9c28b5d8db7f, role_id=86b73ebb032e430c9c08b78a01898cd7)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:21.068575 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b6494b-8eea-48d4-86fe-9af58c2c2d79 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:21.082901 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 472/957] 10.222.0.71 () {58 vars in 1439 bytes} [Mon Jul 4 09:07:20 2022] PUT /identity/v3/projects/4076e316243946e980ce47386b18b28f/users/78668b939c5847b1b73e9c28b5d8db7f/roles/86b73ebb032e430c9c08b78a01898cd7 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:07:21.083494 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b6494b-8eea-48d4-86fe-9af58c2c2d79 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:21.132519 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:21.184279 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8c8f94-a3f4-4e69-9996-324f73f60e70 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:21.228257 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b6494b-8eea-48d4-86fe-9af58c2c2d79 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:21.237681 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd1f632-4772-4aeb-9daa-c50cbfd2f68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:21.238556 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd1f632-4772-4aeb-9daa-c50cbfd2f68b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:21.238948 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd1f632-4772-4aeb-9daa-c50cbfd2f68b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:21.239357 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd1f632-4772-4aeb-9daa-c50cbfd2f68b None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:21.240705 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd1f632-4772-4aeb-9daa-c50cbfd2f68b None admin] RBAC: Adding query filter params (type=compute) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:21.241197 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd1f632-4772-4aeb-9daa-c50cbfd2f68b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:21.243611 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd1f632-4772-4aeb-9daa-c50cbfd2f68b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:21.250652 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 486/958] 10.222.0.71 () {56 vars in 1112 bytes} [Mon Jul 4 09:07:21 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:21.275179 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1ea195-32b1-4e53-9565-26f56f916200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:21.276219 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ea195-32b1-4e53-9565-26f56f916200 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:21.276691 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ea195-32b1-4e53-9565-26f56f916200 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:21.277164 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ea195-32b1-4e53-9565-26f56f916200 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:21.278103 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1ea195-32b1-4e53-9565-26f56f916200 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:21.278593 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1ea195-32b1-4e53-9565-26f56f916200 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:21.280257 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1ea195-32b1-4e53-9565-26f56f916200 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:21.287197 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 473/959] 10.222.0.71 () {56 vars in 1122 bytes} [Mon Jul 4 09:07:21 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:21.293805 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd1f632-4772-4aeb-9daa-c50cbfd2f68b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:21.392248 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:21.415852 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd1f632-4772-4aeb-9daa-c50cbfd2f68b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:21.453666 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c345c9-6450-40f3-9b72-25cd38b40a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:21.454395 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c345c9-6450-40f3-9b72-25cd38b40a58 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:21.454684 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c345c9-6450-40f3-9b72-25cd38b40a58 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:21.454966 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c345c9-6450-40f3-9b72-25cd38b40a58 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:21.455826 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c345c9-6450-40f3-9b72-25cd38b40a58 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:21.457216 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c345c9-6450-40f3-9b72-25cd38b40a58 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:21.479386 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 487/960] 10.222.0.71 () {60 vars in 1144 bytes} [Mon Jul 4 09:07:21 2022] POST /identity/v3/endpoints => generated 349 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:07:22.240208 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e412bad4-3b1f-4538-b1e4-24135a6e6569 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:22.240720 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e412bad4-3b1f-4538-b1e4-24135a6e6569 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:22.240798 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e412bad4-3b1f-4538-b1e4-24135a6e6569 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:22.243211 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 474/961] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:07:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:22.248648 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac78a745-ebac-4be1-b855-5ca35bdbb936 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:22.249109 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac78a745-ebac-4be1-b855-5ca35bdbb936 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:22.249288 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac78a745-ebac-4be1-b855-5ca35bdbb936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:22.266155 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ac78a745-ebac-4be1-b855-5ca35bdbb936 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:22.267341 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ac78a745-ebac-4be1-b855-5ca35bdbb936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:22.353152 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 488/962] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:07:22.371955 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e412bad4-3b1f-4538-b1e4-24135a6e6569 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:22.473913 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:22.479582 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e412bad4-3b1f-4538-b1e4-24135a6e6569 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:22.515216 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cd94c2-5b3c-4fd0-8541-8620143bd016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:22.516191 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cd94c2-5b3c-4fd0-8541-8620143bd016 None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:22.516625 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cd94c2-5b3c-4fd0-8541-8620143bd016 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:22.517071 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cd94c2-5b3c-4fd0-8541-8620143bd016 None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:22.517919 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cd94c2-5b3c-4fd0-8541-8620143bd016 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:22.519488 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cd94c2-5b3c-4fd0-8541-8620143bd016 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:22.534495 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.sql.core [None req-07cd94c2-5b3c-4fd0-8541-8620143bd016 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 04 09:07:22.534495 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 04 09:07:22.534495 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [parameters: {'user_id': '02125c48d6894aeca611c17dff856e6e', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 04 09:07:22.534495 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80171) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 04 09:07:22.543493 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-07cd94c2-5b3c-4fd0-8541-8620143bd016 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 04 09:07:22.543493 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:22.543493 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 04 09:07:22.543493 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 04 09:07:22.543493 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 04 09:07:22.543493 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 04 09:07:22.543493 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 04 09:07:22.543493 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application result = self._query(query) Jul 04 09:07:22.543493 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 04 09:07:22.543493 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application conn.query(q) Jul 04 09:07:22.543493 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 04 09:07:22.543493 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 04 09:07:22.543493 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 04 09:07:22.543493 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application result.read() Jul 04 09:07:22.543493 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 04 09:07:22.544553 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 04 09:07:22.544553 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 04 09:07:22.544553 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 04 09:07:22.544553 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 04 09:07:22.544553 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 04 09:07:22.544553 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 04 09:07:22.544553 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 04 09:07:22.544553 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 04 09:07:22.544553 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:22.544553 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 04 09:07:22.544553 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:22.544553 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:22.544553 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 04 09:07:22.544553 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 04 09:07:22.544553 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 04 09:07:22.544553 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 04 09:07:22.544553 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 04 09:07:22.544553 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application next(self.gen) Jul 04 09:07:22.545808 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 04 09:07:22.545808 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application yield resource Jul 04 09:07:22.545808 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 04 09:07:22.545808 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application next(self.gen) Jul 04 09:07:22.545808 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 04 09:07:22.545808 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self.session.rollback() Jul 04 09:07:22.545808 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 04 09:07:22.545808 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self.force_reraise() Jul 04 09:07:22.545808 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 04 09:07:22.545808 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise self.value Jul 04 09:07:22.545808 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 04 09:07:22.545808 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 04 09:07:22.545808 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 04 09:07:22.545808 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application session.commit() Jul 04 09:07:22.545808 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 04 09:07:22.545808 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 04 09:07:22.545808 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 04 09:07:22.546972 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._prepare_impl() Jul 04 09:07:22.546972 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 04 09:07:22.546972 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self.session.flush() Jul 04 09:07:22.546972 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 04 09:07:22.546972 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._flush(objects) Jul 04 09:07:22.546972 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 04 09:07:22.546972 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 04 09:07:22.546972 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 04 09:07:22.546972 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application compat.raise_( Jul 04 09:07:22.546972 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 04 09:07:22.546972 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception Jul 04 09:07:22.546972 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 04 09:07:22.546972 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application flush_context.execute() Jul 04 09:07:22.546972 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 04 09:07:22.546972 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rec.execute(self) Jul 04 09:07:22.546972 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 04 09:07:22.546972 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 04 09:07:22.548034 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 04 09:07:22.548034 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 04 09:07:22.548034 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 04 09:07:22.548034 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 04 09:07:22.548034 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 04 09:07:22.548034 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 04 09:07:22.548034 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 04 09:07:22.548034 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 04 09:07:22.548034 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 04 09:07:22.548034 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 04 09:07:22.548034 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 04 09:07:22.548034 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 04 09:07:22.548034 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 04 09:07:22.548034 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 04 09:07:22.548034 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 04 09:07:22.548034 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception Jul 04 09:07:22.548943 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 04 09:07:22.548943 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 04 09:07:22.548943 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 04 09:07:22.548943 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 04 09:07:22.548943 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 04 09:07:22.548943 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application result = self._query(query) Jul 04 09:07:22.548943 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 04 09:07:22.548943 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application conn.query(q) Jul 04 09:07:22.548943 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 04 09:07:22.548943 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 04 09:07:22.548943 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 04 09:07:22.548943 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application result.read() Jul 04 09:07:22.548943 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 04 09:07:22.548943 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 04 09:07:22.548943 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 04 09:07:22.548943 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 04 09:07:22.549898 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 04 09:07:22.549898 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 04 09:07:22.549898 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 04 09:07:22.549898 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 04 09:07:22.549898 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 04 09:07:22.549898 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 04 09:07:22.549898 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [parameters: {'user_id': '02125c48d6894aeca611c17dff856e6e', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 04 09:07:22.549898 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 04 09:07:22.549898 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:22.549898 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:07:22.549898 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:22.549898 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:22.549898 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:22.549898 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:22.549898 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:22.551002 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:22.551002 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:22.551002 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:22.551002 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:22.551002 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:22.551002 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:22.551002 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:22.551002 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 04 09:07:22.551002 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 04 09:07:22.551002 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:22.551002 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:22.551002 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:07:22.551002 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:07:22.551002 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:07:22.551002 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:07:22.551002 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 04 09:07:22.551002 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 04 09:07:22.552209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 04 09:07:22.552209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 04 09:07:22.552209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 04 09:07:22.552209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 04 09:07:22.552209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 04 09:07:22.552209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:22.559134 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 475/963] 10.222.0.71 () {60 vars in 1131 bytes} [Mon Jul 4 09:07:22 2022] POST /identity/v3/users => generated 160 bytes in 183 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:22.561078 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac78a745-ebac-4be1-b855-5ca35bdbb936 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:22.628219 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:22.667423 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac78a745-ebac-4be1-b855-5ca35bdbb936 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:22.704454 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cbd419-7489-437e-8402-2a729f6b2fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:22.705609 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cbd419-7489-437e-8402-2a729f6b2fcd None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:22.706058 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cbd419-7489-437e-8402-2a729f6b2fcd None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:22.706529 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cbd419-7489-437e-8402-2a729f6b2fcd None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:22.715229 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cbd419-7489-437e-8402-2a729f6b2fcd None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:22.716850 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cbd419-7489-437e-8402-2a729f6b2fcd None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:22.731634 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-b6cbd419-7489-437e-8402-2a729f6b2fcd None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 04 09:07:22.731634 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:22.731634 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:22.731634 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:22.731634 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:22.731634 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:22.731634 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:22.731634 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:22.731634 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:22.731634 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:22.731634 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:22.731634 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:22.731634 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:07:22.731634 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:07:22.731634 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:07:22.731634 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:07:22.732784 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:22.732784 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:22.732784 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:07:22.732784 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:07:22.732784 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:07:22.732784 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:07:22.732784 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:22.732784 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:22.732784 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:22.732784 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:22.732784 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:22.732784 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:22.732784 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:22.732784 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:22.732784 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:22.732784 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:22.732784 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:22.732784 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:22.733998 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:22.733998 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:22.733998 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:07:22.733998 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:07:22.733998 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:07:22.733998 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:07:22.733998 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:07:22.733998 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:07:22.733998 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 04 09:07:22.733998 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:07:22.740727 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 489/964] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:07:22 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:22.742657 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cd94c2-5b3c-4fd0-8541-8620143bd016 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:22.848931 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cd94c2-5b3c-4fd0-8541-8620143bd016 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:22.884536 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e5444b-fc56-4cbc-9ffe-e68a414501f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:22.885533 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e5444b-fc56-4cbc-9ffe-e68a414501f0 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:22.886018 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e5444b-fc56-4cbc-9ffe-e68a414501f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:22.886470 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e5444b-fc56-4cbc-9ffe-e68a414501f0 None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:22.887437 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e5444b-fc56-4cbc-9ffe-e68a414501f0 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:22.887891 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e5444b-fc56-4cbc-9ffe-e68a414501f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:22.889705 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e5444b-fc56-4cbc-9ffe-e68a414501f0 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:22.933175 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 476/965] 10.222.0.71 () {56 vars in 1101 bytes} [Mon Jul 4 09:07:22 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:23.309222 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992df5e9-6d57-4330-bc94-fdfbc4331cdf None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:23.309775 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992df5e9-6d57-4330-bc94-fdfbc4331cdf None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:23.309775 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992df5e9-6d57-4330-bc94-fdfbc4331cdf None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:23.311350 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 490/966] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:07:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:23.316411 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f094b3e-4c28-4f98-ad55-4931e4386c91 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:23.316832 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f094b3e-4c28-4f98-ad55-4931e4386c91 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:23.316985 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f094b3e-4c28-4f98-ad55-4931e4386c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:23.334733 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-2f094b3e-4c28-4f98-ad55-4931e4386c91 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:23.336828 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-2f094b3e-4c28-4f98-ad55-4931e4386c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:23.351906 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:23.385664 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959e10f-7c56-44fa-80c8-4bafde8bb733 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:23.385996 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959e10f-7c56-44fa-80c8-4bafde8bb733 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:23.386240 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959e10f-7c56-44fa-80c8-4bafde8bb733 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:23.387846 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 491/967] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:07:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:07:23.392826 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce3f1e-9002-445c-ad4a-b321bf6247d1 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:23.393141 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce3f1e-9002-445c-ad4a-b321bf6247d1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:23.393426 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce3f1e-9002-445c-ad4a-b321bf6247d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:23.411833 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d3ce3f1e-9002-445c-ad4a-b321bf6247d1 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:23.413911 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d3ce3f1e-9002-445c-ad4a-b321bf6247d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:23.416940 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 477/968] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:07:23.440736 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f094b3e-4c28-4f98-ad55-4931e4386c91 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:23.482383 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 492/969] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:07:23.505955 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ce3f1e-9002-445c-ad4a-b321bf6247d1 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:23.558872 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f094b3e-4c28-4f98-ad55-4931e4386c91 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:23.564838 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:23.596141 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017daeb6-ea25-48ee-9a1a-33d0c5ee101d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:23.596963 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017daeb6-ea25-48ee-9a1a-33d0c5ee101d None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:23.597278 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017daeb6-ea25-48ee-9a1a-33d0c5ee101d None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:23.597558 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017daeb6-ea25-48ee-9a1a-33d0c5ee101d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:23.608957 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017daeb6-ea25-48ee-9a1a-33d0c5ee101d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:07:23.614178 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ce3f1e-9002-445c-ad4a-b321bf6247d1 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:23.618188 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-017daeb6-ea25-48ee-9a1a-33d0c5ee101d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 09:07:23.618188 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:23.618188 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:23.618188 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:23.618188 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:23.618188 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:23.618188 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:23.618188 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:23.618188 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:23.618188 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:23.618188 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:23.618188 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:23.618188 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:07:23.618188 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:07:23.618188 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:07:23.618188 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:07:23.619308 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:07:23.619308 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:07:23.619308 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:07:23.619308 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:07:23.619308 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:23.619308 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:23.619308 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:23.619308 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:23.619308 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:23.619308 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:23.619308 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:23.619308 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:23.619308 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:23.619308 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:23.619308 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:23.619308 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:23.620580 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:23.620580 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:23.620580 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:23.620580 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:23.620580 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:07:23.620580 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:07:23.620580 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:07:23.620580 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:07:23.620580 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:07:23.620580 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:07:23.620580 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 09:07:23.620580 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:23.620580 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 478/970] 10.222.0.71 () {56 vars in 1120 bytes} [Mon Jul 4 09:07:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:23.624774 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017daeb6-ea25-48ee-9a1a-33d0c5ee101d None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:23.636572 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:23.652369 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b559bc-2aea-4b83-bd51-46ac6b0e723d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:23.653059 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b559bc-2aea-4b83-bd51-46ac6b0e723d None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:23.653360 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b559bc-2aea-4b83-bd51-46ac6b0e723d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:23.653640 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b559bc-2aea-4b83-bd51-46ac6b0e723d None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:23.654361 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b559bc-2aea-4b83-bd51-46ac6b0e723d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:23.655719 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b559bc-2aea-4b83-bd51-46ac6b0e723d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:23.670908 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 493/971] 10.222.0.71 () {60 vars in 1141 bytes} [Mon Jul 4 09:07:23 2022] POST /identity/v3/services => generated 242 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:07:23.677847 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada63120-401f-411d-954e-97058a495ece None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:23.677847 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada63120-401f-411d-954e-97058a495ece None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:23.677847 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada63120-401f-411d-954e-97058a495ece None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:23.677847 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 494/972] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:07:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:23.680370 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a693da1b-d0f0-4f59-b0d8-5ff3e9b88a4b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:23.680663 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a693da1b-d0f0-4f59-b0d8-5ff3e9b88a4b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:23.680954 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a693da1b-d0f0-4f59-b0d8-5ff3e9b88a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:23.699424 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a693da1b-d0f0-4f59-b0d8-5ff3e9b88a4b None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:23.700726 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a693da1b-d0f0-4f59-b0d8-5ff3e9b88a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:23.743571 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017daeb6-ea25-48ee-9a1a-33d0c5ee101d None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:23.790416 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843d37fb-0e1c-4eab-92e5-2759dff7a323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:23.790643 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 495/973] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:23 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:07:23.791631 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843d37fb-0e1c-4eab-92e5-2759dff7a323 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:23.792198 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843d37fb-0e1c-4eab-92e5-2759dff7a323 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:23.792624 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843d37fb-0e1c-4eab-92e5-2759dff7a323 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:23.793579 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843d37fb-0e1c-4eab-92e5-2759dff7a323 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:23.794046 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843d37fb-0e1c-4eab-92e5-2759dff7a323 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:23.795602 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843d37fb-0e1c-4eab-92e5-2759dff7a323 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:23.805666 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 479/974] 10.222.0.71 () {56 vars in 1115 bytes} [Mon Jul 4 09:07:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:07:23.811350 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a693da1b-d0f0-4f59-b0d8-5ff3e9b88a4b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:23.813901 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843d37fb-0e1c-4eab-92e5-2759dff7a323 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:23.866920 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:23.889874 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:23.930436 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a693da1b-d0f0-4f59-b0d8-5ff3e9b88a4b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:23.935376 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843d37fb-0e1c-4eab-92e5-2759dff7a323 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:23.967356 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474363c2-87ef-4835-8ab6-c878aeae57fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:23.968534 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474363c2-87ef-4835-8ab6-c878aeae57fa None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:23.969020 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474363c2-87ef-4835-8ab6-c878aeae57fa None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:23.969331 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474363c2-87ef-4835-8ab6-c878aeae57fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:23.975370 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b23d833-6ebf-4346-bd33-28f903a8ccd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:23.976772 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b23d833-6ebf-4346-bd33-28f903a8ccd7 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:23.977275 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b23d833-6ebf-4346-bd33-28f903a8ccd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:23.977382 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b23d833-6ebf-4346-bd33-28f903a8ccd7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:23.978009 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b23d833-6ebf-4346-bd33-28f903a8ccd7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:23.979387 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b23d833-6ebf-4346-bd33-28f903a8ccd7 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:23.980707 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474363c2-87ef-4835-8ab6-c878aeae57fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:23.982168 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474363c2-87ef-4835-8ab6-c878aeae57fa None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:23.992569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-3b23d833-6ebf-4346-bd33-28f903a8ccd7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 09:07:23.992569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:23.992569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:07:23.992569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:07:23.992569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:07:23.992569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:07:23.992569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:07:23.992569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:07:23.992569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 09:07:23.992569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:23.992569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:07:23.992569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:23.992569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:23.992569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:23.992569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:23.992569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:23.992569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:23.993687 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:23.993687 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:23.993687 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:23.993687 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:23.993687 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:23.993687 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:23.993687 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:07:23.993687 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:07:23.993687 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:07:23.993687 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:07:23.993687 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:23.993687 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:23.993687 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:23.993687 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:23.993687 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:23.993687 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:23.994546 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:23.994546 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:23.994546 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:23.994546 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:23.994546 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:23.994546 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:23.994546 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:23.994546 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:23.994546 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:23.994546 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:23.994546 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:07:23.994546 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:07:23.994546 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 09:07:23.994546 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:23.994546 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 480/975] 10.222.0.71 () {56 vars in 1126 bytes} [Mon Jul 4 09:07:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:07:23.999597 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b23d833-6ebf-4346-bd33-28f903a8ccd7 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:24.004120 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-474363c2-87ef-4835-8ab6-c878aeae57fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:07:24.004120 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:24.004120 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:24.004120 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:24.004120 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:24.004120 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:24.004120 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:24.004120 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:24.004120 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:24.004120 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:24.004120 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:24.004120 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:24.004120 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:07:24.004120 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:07:24.004120 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:07:24.005010 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:07:24.005010 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:24.005010 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:24.005010 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:24.005010 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:24.005010 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:24.005010 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:24.005010 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:24.005010 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:24.005010 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:24.005010 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:24.005010 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:24.005010 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:24.005010 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:24.005010 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:24.005010 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:24.005010 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:24.013261 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:07:24.013261 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:07:24.013261 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:07:24.013261 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:07:24.013261 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 496/976] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:07:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:07:24.013261 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474363c2-87ef-4835-8ab6-c878aeae57fa None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:24.113139 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:24.114230 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b23d833-6ebf-4346-bd33-28f903a8ccd7 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:24.123596 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474363c2-87ef-4835-8ab6-c878aeae57fa None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:24.146106 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:24.154207 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4500735-6f35-4bb3-b098-dd33723676a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:24.155035 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4500735-6f35-4bb3-b098-dd33723676a9 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:24.155424 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4500735-6f35-4bb3-b098-dd33723676a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:24.155801 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4500735-6f35-4bb3-b098-dd33723676a9 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:24.156620 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4500735-6f35-4bb3-b098-dd33723676a9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:24.157031 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4500735-6f35-4bb3-b098-dd33723676a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:24.158550 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4500735-6f35-4bb3-b098-dd33723676a9 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:24.164702 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 481/977] 10.222.0.71 () {56 vars in 1122 bytes} [Mon Jul 4 09:07:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:07:24.170128 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4500735-6f35-4bb3-b098-dd33723676a9 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:24.174407 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adb2629-2f9d-4cc2-b4ad-7f5c6aff077e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:24.175200 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb2629-2f9d-4cc2-b4ad-7f5c6aff077e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:24.175578 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb2629-2f9d-4cc2-b4ad-7f5c6aff077e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:24.175957 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb2629-2f9d-4cc2-b4ad-7f5c6aff077e None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:24.176787 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adb2629-2f9d-4cc2-b4ad-7f5c6aff077e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:24.177302 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adb2629-2f9d-4cc2-b4ad-7f5c6aff077e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:24.179521 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adb2629-2f9d-4cc2-b4ad-7f5c6aff077e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:24.196237 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 497/978] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:07:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:24.197790 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adb2629-2f9d-4cc2-b4ad-7f5c6aff077e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:24.279884 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4500735-6f35-4bb3-b098-dd33723676a9 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:24.306608 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adb2629-2f9d-4cc2-b4ad-7f5c6aff077e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:24.315371 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec82d697-4b9f-431b-ad3c-fe502a599031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:24.316043 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec82d697-4b9f-431b-ad3c-fe502a599031 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:24.316341 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec82d697-4b9f-431b-ad3c-fe502a599031 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:24.316620 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec82d697-4b9f-431b-ad3c-fe502a599031 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:24.317367 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec82d697-4b9f-431b-ad3c-fe502a599031 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:24.317679 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec82d697-4b9f-431b-ad3c-fe502a599031 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:24.319059 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec82d697-4b9f-431b-ad3c-fe502a599031 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:24.324742 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 482/979] 10.222.0.71 () {56 vars in 1122 bytes} [Mon Jul 4 09:07:24 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:07:24.330385 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec82d697-4b9f-431b-ad3c-fe502a599031 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:24.349780 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f52fe9-0b8a-4a1a-bb02-69e61b979a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:24.350663 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f52fe9-0b8a-4a1a-bb02-69e61b979a26 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:24.350919 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f52fe9-0b8a-4a1a-bb02-69e61b979a26 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:24.351196 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f52fe9-0b8a-4a1a-bb02-69e61b979a26 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:24.358862 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f52fe9-0b8a-4a1a-bb02-69e61b979a26 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:24.360290 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f52fe9-0b8a-4a1a-bb02-69e61b979a26 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:24.375431 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-48f52fe9-0b8a-4a1a-bb02-69e61b979a26 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 09:07:24.375431 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:24.375431 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:24.375431 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:24.375431 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:24.375431 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:24.375431 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:24.375431 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:24.375431 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:24.375431 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:24.375431 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:24.375431 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:24.375431 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:07:24.375431 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:07:24.375431 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:07:24.375431 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:07:24.376432 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:24.376432 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:24.376432 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:07:24.376432 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:07:24.376432 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:07:24.376432 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:07:24.376432 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:24.376432 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:24.376432 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:24.376432 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:24.376432 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:24.376432 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:24.376432 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:24.376432 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:24.376432 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:24.376432 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:24.376432 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:24.376432 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:24.377889 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:24.377889 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:24.377889 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:07:24.377889 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:07:24.377889 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:07:24.377889 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:07:24.377889 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:07:24.377889 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:07:24.377889 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 09:07:24.377889 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:07:24.377889 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 498/980] 10.222.0.71 () {56 vars in 1117 bytes} [Mon Jul 4 09:07:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:24.383023 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f52fe9-0b8a-4a1a-bb02-69e61b979a26 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:24.400196 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:24.400876 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:24.447905 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec82d697-4b9f-431b-ad3c-fe502a599031 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:24.483695 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dea796-c82e-478a-a711-5b33720a1b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:24.484450 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dea796-c82e-478a-a711-5b33720a1b8f None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:24.484809 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dea796-c82e-478a-a711-5b33720a1b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:24.485187 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dea796-c82e-478a-a711-5b33720a1b8f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:24.486064 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dea796-c82e-478a-a711-5b33720a1b8f None admin] RBAC: Adding query filter params (interface=admin, service_id=fd599f217ead4e98aeac1954f00c098b, region_id=RegionOne) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:24.486451 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dea796-c82e-478a-a711-5b33720a1b8f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:24.487911 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dea796-c82e-478a-a711-5b33720a1b8f None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:24.493885 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f52fe9-0b8a-4a1a-bb02-69e61b979a26 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:24.494475 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 483/981] 10.222.0.71 () {56 vars in 1249 bytes} [Mon Jul 4 09:07:24 2022] GET /identity/v3/endpoints?service_id=fd599f217ead4e98aeac1954f00c098b&interface=admin®ion_id=RegionOne => generated 193 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:07:24.499667 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dea796-c82e-478a-a711-5b33720a1b8f None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:24.530903 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0494493-08f1-4d47-b62b-4472f117393c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:24.531606 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0494493-08f1-4d47-b62b-4472f117393c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:24.531890 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0494493-08f1-4d47-b62b-4472f117393c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:24.532164 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0494493-08f1-4d47-b62b-4472f117393c None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:24.533002 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0494493-08f1-4d47-b62b-4472f117393c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:24.533350 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0494493-08f1-4d47-b62b-4472f117393c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:24.534772 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0494493-08f1-4d47-b62b-4472f117393c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:24.554867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 499/982] 10.222.0.71 () {56 vars in 1149 bytes} [Mon Jul 4 09:07:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:07:24.560290 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0494493-08f1-4d47-b62b-4472f117393c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:24.619475 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dea796-c82e-478a-a711-5b33720a1b8f None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:24.635541 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:24.642835 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:24.656130 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb143c4e-2073-4d3e-a13b-367b67407e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:24.656793 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb143c4e-2073-4d3e-a13b-367b67407e99 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:24.657118 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb143c4e-2073-4d3e-a13b-367b67407e99 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:24.657379 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb143c4e-2073-4d3e-a13b-367b67407e99 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:24.658057 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb143c4e-2073-4d3e-a13b-367b67407e99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:24.659428 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb143c4e-2073-4d3e-a13b-367b67407e99 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:24.666984 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 484/983] 10.222.0.71 () {56 vars in 1087 bytes} [Mon Jul 4 09:07:24 2022] GET /identity/v3/services => generated 1727 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:07:24.684730 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0494493-08f1-4d47-b62b-4472f117393c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:24.724288 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f13f076-7fb0-4de1-8c3a-7f0986340511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:24.724962 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13f076-7fb0-4de1-8c3a-7f0986340511 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:24.725282 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13f076-7fb0-4de1-8c3a-7f0986340511 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:24.726212 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13f076-7fb0-4de1-8c3a-7f0986340511 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:24.804400 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f13f076-7fb0-4de1-8c3a-7f0986340511 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:24.805933 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f13f076-7fb0-4de1-8c3a-7f0986340511 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:24.884805 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:24.918686 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 500/984] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:07:24 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 04 09:07:24.923963 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb143c4e-2073-4d3e-a13b-367b67407e99 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:25.026057 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb143c4e-2073-4d3e-a13b-367b67407e99 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:25.059188 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bec532-e661-4c25-8f1a-80b0ad60d3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:25.059982 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bec532-e661-4c25-8f1a-80b0ad60d3e6 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:25.060264 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bec532-e661-4c25-8f1a-80b0ad60d3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:25.060543 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bec532-e661-4c25-8f1a-80b0ad60d3e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:25.071525 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bec532-e661-4c25-8f1a-80b0ad60d3e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:25.072974 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bec532-e661-4c25-8f1a-80b0ad60d3e6 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:25.091933 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-e1bec532-e661-4c25-8f1a-80b0ad60d3e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:07:25.091933 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:25.091933 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:25.091933 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:25.091933 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:25.091933 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:25.091933 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:25.091933 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:25.091933 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:25.091933 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:25.091933 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:25.091933 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:25.091933 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:07:25.091933 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:07:25.091933 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:07:25.092902 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:07:25.092902 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:25.092902 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:25.092902 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:25.092902 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:25.092902 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:25.092902 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:25.092902 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:25.092902 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:25.092902 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:25.092902 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:25.092902 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:25.092902 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:25.092902 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:25.092902 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:25.092902 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:25.092902 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:25.094433 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:07:25.094433 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:07:25.094433 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:07:25.094433 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:25.097111 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 485/985] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:07:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:25.098792 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f13f076-7fb0-4de1-8c3a-7f0986340511 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:25.212603 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f13f076-7fb0-4de1-8c3a-7f0986340511 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:25.246898 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c57b30-5daf-42ce-ad39-29147387cfcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:25.247709 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c57b30-5daf-42ce-ad39-29147387cfcb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:25.248086 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c57b30-5daf-42ce-ad39-29147387cfcb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:25.248456 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c57b30-5daf-42ce-ad39-29147387cfcb None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:25.249326 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c57b30-5daf-42ce-ad39-29147387cfcb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:25.249746 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c57b30-5daf-42ce-ad39-29147387cfcb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:25.251176 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c57b30-5daf-42ce-ad39-29147387cfcb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:25.263559 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 501/986] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:07:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:07:25.269083 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bec532-e661-4c25-8f1a-80b0ad60d3e6 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:25.313211 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:25.379736 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bec532-e661-4c25-8f1a-80b0ad60d3e6 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:25.417399 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75808fa4-9d3f-4265-8013-10098eec7828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:25.418205 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75808fa4-9d3f-4265-8013-10098eec7828 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:25.418490 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75808fa4-9d3f-4265-8013-10098eec7828 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:25.418767 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75808fa4-9d3f-4265-8013-10098eec7828 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:25.425935 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75808fa4-9d3f-4265-8013-10098eec7828 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:25.427393 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75808fa4-9d3f-4265-8013-10098eec7828 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:25.442699 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-75808fa4-9d3f-4265-8013-10098eec7828 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:07:25.442699 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:25.442699 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:25.442699 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:25.442699 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:25.442699 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:25.442699 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:25.442699 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:25.442699 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:25.442699 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:25.442699 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:25.442699 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:25.442699 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:07:25.442699 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:07:25.442699 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:07:25.443640 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:07:25.443640 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:25.443640 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:25.443640 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:25.443640 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:25.443640 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:25.443640 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:25.443640 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:25.443640 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:25.443640 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:25.443640 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:25.443640 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:25.443640 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:25.443640 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:25.443640 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:25.443640 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:25.443640 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:25.445271 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:07:25.445271 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:07:25.445271 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:07:25.445271 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:07:25.445271 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:07:25.445271 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:07:25.445271 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:07:25.445271 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:25.449134 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 486/987] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 4 09:07:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:25.451730 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c57b30-5daf-42ce-ad39-29147387cfcb None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:25.485661 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:25.558244 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c57b30-5daf-42ce-ad39-29147387cfcb None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:25.592459 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9b914b-aedb-4b4a-8ec3-ba6c18665f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:25.593292 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b914b-aedb-4b4a-8ec3-ba6c18665f94 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:25.593704 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b914b-aedb-4b4a-8ec3-ba6c18665f94 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:25.594088 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b914b-aedb-4b4a-8ec3-ba6c18665f94 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:25.594983 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9b914b-aedb-4b4a-8ec3-ba6c18665f94 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:25.595399 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9b914b-aedb-4b4a-8ec3-ba6c18665f94 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:25.596939 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9b914b-aedb-4b4a-8ec3-ba6c18665f94 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:25.609466 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 502/988] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 4 09:07:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:25.615100 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75808fa4-9d3f-4265-8013-10098eec7828 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:25.714888 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:25.720138 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75808fa4-9d3f-4265-8013-10098eec7828 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:25.731014 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6983df-b7e1-4a45-82af-844c00461428 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:25.731302 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6983df-b7e1-4a45-82af-844c00461428 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:25.731576 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6983df-b7e1-4a45-82af-844c00461428 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:25.733094 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 503/989] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:07:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:07:25.737725 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e4c00-5cc3-41df-bede-c5ad569cfe34 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:25.737958 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e4c00-5cc3-41df-bede-c5ad569cfe34 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:25.738233 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e4c00-5cc3-41df-bede-c5ad569cfe34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:25.756581 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-599e4c00-5cc3-41df-bede-c5ad569cfe34 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:25.757954 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-599e4c00-5cc3-41df-bede-c5ad569cfe34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:25.760406 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64a62db-d519-458e-9a72-d8aea0aad9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:25.761173 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a62db-d519-458e-9a72-d8aea0aad9ed None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:25.761530 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a62db-d519-458e-9a72-d8aea0aad9ed None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:25.761880 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a62db-d519-458e-9a72-d8aea0aad9ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:25.762804 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64a62db-d519-458e-9a72-d8aea0aad9ed None admin] RBAC: Adding query filter params (role.id=86b73ebb032e430c9c08b78a01898cd7, scope.project.id=4076e316243946e980ce47386b18b28f, user.id=78668b939c5847b1b73e9c28b5d8db7f) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:25.763196 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64a62db-d519-458e-9a72-d8aea0aad9ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:25.764713 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64a62db-d519-458e-9a72-d8aea0aad9ed None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:25.781094 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 487/990] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 4 09:07:25 2022] GET /identity/v3/role_assignments?user.id=78668b939c5847b1b73e9c28b5d8db7f&scope.project.id=4076e316243946e980ce47386b18b28f&role.id=86b73ebb032e430c9c08b78a01898cd7 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:25.844709 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 504/991] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:25 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:07:25.867779 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64a62db-d519-458e-9a72-d8aea0aad9ed None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:25.979379 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64a62db-d519-458e-9a72-d8aea0aad9ed None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:26.015474 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7515ac9d-b74d-49c5-a11d-b81371998d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:26.016201 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7515ac9d-b74d-49c5-a11d-b81371998d4f None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:26.016489 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7515ac9d-b74d-49c5-a11d-b81371998d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:26.016770 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7515ac9d-b74d-49c5-a11d-b81371998d4f None admin] PATH_INFO: `/v3/services/compute` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:26.017449 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7515ac9d-b74d-49c5-a11d-b81371998d4f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:26.018821 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7515ac9d-b74d-49c5-a11d-b81371998d4f None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:26.031266 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-7515ac9d-b74d-49c5-a11d-b81371998d4f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 04 09:07:26.031266 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:26.031266 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:07:26.031266 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:07:26.031266 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:07:26.031266 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:07:26.031266 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:07:26.031266 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:07:26.031266 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 04 09:07:26.031266 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:26.031266 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:07:26.031266 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:26.031266 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:26.031266 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:26.031266 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:26.031266 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:26.031266 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:26.032260 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:26.032260 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:26.032260 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:26.032260 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:26.032260 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:26.032260 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:26.032260 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:07:26.032260 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:07:26.032260 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:07:26.032260 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:07:26.032260 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:26.032260 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:26.032260 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:26.032260 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:26.032260 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:26.032260 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:26.033151 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:26.033151 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:26.033151 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:26.033151 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:26.033151 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:26.033151 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:26.033151 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:26.033151 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:26.033151 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:26.033151 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:26.033151 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:07:26.033151 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:07:26.033151 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 04 09:07:26.033151 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:26.033151 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 488/992] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 4 09:07:25 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:26.038472 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599e4c00-5cc3-41df-bede-c5ad569cfe34 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:26.063353 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:26.142600 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599e4c00-5cc3-41df-bede-c5ad569cfe34 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:26.177891 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ad8b62-750a-4aeb-83e2-b46b75024da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:26.178592 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad8b62-750a-4aeb-83e2-b46b75024da9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:26.178918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad8b62-750a-4aeb-83e2-b46b75024da9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:26.179285 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad8b62-750a-4aeb-83e2-b46b75024da9 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:26.179994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ad8b62-750a-4aeb-83e2-b46b75024da9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:26.180302 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ad8b62-750a-4aeb-83e2-b46b75024da9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:26.181698 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ad8b62-750a-4aeb-83e2-b46b75024da9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:26.186707 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 505/993] 10.222.0.71 () {56 vars in 1112 bytes} [Mon Jul 4 09:07:26 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:26.192095 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7515ac9d-b74d-49c5-a11d-b81371998d4f None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:26.221437 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:26.298163 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7515ac9d-b74d-49c5-a11d-b81371998d4f None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:26.331923 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a1eaa4-3747-48d2-bdf8-77f80a57063c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:26.332759 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a1eaa4-3747-48d2-bdf8-77f80a57063c None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:26.333146 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a1eaa4-3747-48d2-bdf8-77f80a57063c None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:26.333529 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a1eaa4-3747-48d2-bdf8-77f80a57063c None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:26.334361 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a1eaa4-3747-48d2-bdf8-77f80a57063c None admin] RBAC: Adding query filter params (type=compute) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:26.334775 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a1eaa4-3747-48d2-bdf8-77f80a57063c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:26.336778 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a1eaa4-3747-48d2-bdf8-77f80a57063c None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:26.347181 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 489/994] 10.222.0.71 () {56 vars in 1112 bytes} [Mon Jul 4 09:07:26 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:26.348251 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ad8b62-750a-4aeb-83e2-b46b75024da9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:26.443915 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:26.460891 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ad8b62-750a-4aeb-83e2-b46b75024da9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:26.496172 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1a227b-8c7b-4d04-9403-c66a1cdd858e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:26.496836 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a227b-8c7b-4d04-9403-c66a1cdd858e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:26.497152 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a227b-8c7b-4d04-9403-c66a1cdd858e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:26.497435 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a227b-8c7b-4d04-9403-c66a1cdd858e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:26.498226 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1a227b-8c7b-4d04-9403-c66a1cdd858e None admin] RBAC: Adding query filter params (interface=public, service_id=34972bf57d2a4d68ae0497d2e2bcf5f1, region_id=RegionOne) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:26.498536 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1a227b-8c7b-4d04-9403-c66a1cdd858e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:26.499901 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1a227b-8c7b-4d04-9403-c66a1cdd858e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:26.509618 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 506/995] 10.222.0.71 () {56 vars in 1251 bytes} [Mon Jul 4 09:07:26 2022] GET /identity/v3/endpoints?service_id=34972bf57d2a4d68ae0497d2e2bcf5f1&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:07:26.510769 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a1eaa4-3747-48d2-bdf8-77f80a57063c None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:26.603741 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:26.614257 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a1eaa4-3747-48d2-bdf8-77f80a57063c None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:26.649626 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae187c5f-0d62-4f21-bf6c-d0b8263540de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:26.650517 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae187c5f-0d62-4f21-bf6c-d0b8263540de None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:26.650951 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae187c5f-0d62-4f21-bf6c-d0b8263540de None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:26.651405 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae187c5f-0d62-4f21-bf6c-d0b8263540de None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:26.652245 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae187c5f-0d62-4f21-bf6c-d0b8263540de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:26.653786 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae187c5f-0d62-4f21-bf6c-d0b8263540de None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:26.665069 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 490/996] 10.222.0.71 () {56 vars in 1087 bytes} [Mon Jul 4 09:07:26 2022] GET /identity/v3/services => generated 1727 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:07:26.724518 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee01dc-7fdb-4644-867d-aa347a011957 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:26.724812 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee01dc-7fdb-4644-867d-aa347a011957 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:26.725127 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee01dc-7fdb-4644-867d-aa347a011957 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:26.726563 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 507/997] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:07:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:26.732301 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3d106-d6c7-4d66-84b4-1c06dd8c9f53 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:26.732591 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3d106-d6c7-4d66-84b4-1c06dd8c9f53 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:26.732877 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3d106-d6c7-4d66-84b4-1c06dd8c9f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:26.751063 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-3de3d106-d6c7-4d66-84b4-1c06dd8c9f53 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:26.752301 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-3de3d106-d6c7-4d66-84b4-1c06dd8c9f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:26.827149 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 491/998] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:26 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:07:26.850239 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ee01dc-7fdb-4644-867d-aa347a011957 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:26.959625 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ee01dc-7fdb-4644-867d-aa347a011957 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:26.995792 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec0ca0e-5a6d-4540-83bd-002533044ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:26.996610 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec0ca0e-5a6d-4540-83bd-002533044ced None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:26.996898 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec0ca0e-5a6d-4540-83bd-002533044ced None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:26.997212 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec0ca0e-5a6d-4540-83bd-002533044ced None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:26.997872 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec0ca0e-5a6d-4540-83bd-002533044ced None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:26.999253 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec0ca0e-5a6d-4540-83bd-002533044ced None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:27.012159 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-cec0ca0e-5a6d-4540-83bd-002533044ced None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 09:07:27.012159 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:27.012159 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:07:27.012159 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:07:27.012159 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:07:27.012159 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:07:27.012159 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:07:27.012159 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:07:27.012159 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 09:07:27.012159 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:07:27.012159 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:07:27.012159 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:07:27.012159 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:27.012159 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:27.012159 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:27.012159 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:27.012159 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:27.013486 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:27.013486 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:27.013486 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:27.013486 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:27.013486 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:27.013486 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:27.013486 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:07:27.013486 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:07:27.013486 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:07:27.013486 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:07:27.013486 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:27.013486 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:27.013486 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:27.013486 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:27.013486 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:27.013486 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:27.014331 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:27.014331 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:27.014331 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:27.014331 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:27.014331 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:27.014331 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:27.014331 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:27.014331 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:27.014331 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:27.014331 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:27.014331 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:07:27.014331 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:07:27.014331 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 09:07:27.014331 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:07:27.014331 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 508/999] 10.222.0.71 () {56 vars in 1126 bytes} [Mon Jul 4 09:07:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:27.019295 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de3d106-d6c7-4d66-84b4-1c06dd8c9f53 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:27.106625 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:27.124298 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de3d106-d6c7-4d66-84b4-1c06dd8c9f53 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:27.158832 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a858d53a-036a-4a21-a8ba-7661fbc896a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:27.159505 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a858d53a-036a-4a21-a8ba-7661fbc896a6 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:27.159790 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a858d53a-036a-4a21-a8ba-7661fbc896a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:27.160068 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a858d53a-036a-4a21-a8ba-7661fbc896a6 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:27.160781 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a858d53a-036a-4a21-a8ba-7661fbc896a6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:27.161121 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a858d53a-036a-4a21-a8ba-7661fbc896a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:27.162497 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a858d53a-036a-4a21-a8ba-7661fbc896a6 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:27.169229 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 492/1000] 10.222.0.71 () {56 vars in 1122 bytes} [Mon Jul 4 09:07:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:27.174092 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec0ca0e-5a6d-4540-83bd-002533044ced None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:27.191025 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:27.276774 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec0ca0e-5a6d-4540-83bd-002533044ced None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:27.311384 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886c6676-8e69-4ebb-82d9-2636b0d74380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:27.312244 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c6676-8e69-4ebb-82d9-2636b0d74380 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:27.312635 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c6676-8e69-4ebb-82d9-2636b0d74380 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:27.313015 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c6676-8e69-4ebb-82d9-2636b0d74380 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:27.313867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886c6676-8e69-4ebb-82d9-2636b0d74380 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:27.314278 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886c6676-8e69-4ebb-82d9-2636b0d74380 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:27.315804 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886c6676-8e69-4ebb-82d9-2636b0d74380 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:27.321194 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 509/1001] 10.222.0.71 () {56 vars in 1122 bytes} [Mon Jul 4 09:07:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:27.326524 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a858d53a-036a-4a21-a8ba-7661fbc896a6 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:27.434628 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a858d53a-036a-4a21-a8ba-7661fbc896a6 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:27.468854 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfc8441-54a4-4541-a358-814f1bb36591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:27.469567 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc8441-54a4-4541-a358-814f1bb36591 None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:27.469905 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc8441-54a4-4541-a358-814f1bb36591 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:27.470186 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc8441-54a4-4541-a358-814f1bb36591 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:27.470923 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfc8441-54a4-4541-a358-814f1bb36591 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:27.472280 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfc8441-54a4-4541-a358-814f1bb36591 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:27.497086 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 493/1002] 10.222.0.71 () {60 vars in 1144 bytes} [Mon Jul 4 09:07:27 2022] POST /identity/v3/endpoints => generated 325 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:07:27.872707 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f6a5a-761d-4106-a055-86dda04ca58d None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:27.873260 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f6a5a-761d-4106-a055-86dda04ca58d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:27.873339 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f6a5a-761d-4106-a055-86dda04ca58d None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:27.874827 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 510/1003] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:07:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:27.879677 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e3c31-46d7-40b0-90a4-92d8dbef496f None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:27.880077 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e3c31-46d7-40b0-90a4-92d8dbef496f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:27.880275 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e3c31-46d7-40b0-90a4-92d8dbef496f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:27.896616 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-135e3c31-46d7-40b0-90a4-92d8dbef496f None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:27.898367 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-135e3c31-46d7-40b0-90a4-92d8dbef496f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:27.900397 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:27.977656 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 494/1004] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:27 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:07:27.999944 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847f6a5a-761d-4106-a055-86dda04ca58d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:28.083489 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:28.107992 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847f6a5a-761d-4106-a055-86dda04ca58d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:28.142327 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ead0d1-94c8-4029-89f4-74ebc3501c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:28.143040 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ead0d1-94c8-4029-89f4-74ebc3501c9a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:28.143329 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ead0d1-94c8-4029-89f4-74ebc3501c9a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:28.143608 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ead0d1-94c8-4029-89f4-74ebc3501c9a None admin] PATH_INFO: `/v3/services/image` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:28.144374 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ead0d1-94c8-4029-89f4-74ebc3501c9a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:28.145786 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ead0d1-94c8-4029-89f4-74ebc3501c9a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:28.158171 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-26ead0d1-94c8-4029-89f4-74ebc3501c9a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 04 09:07:28.158171 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:28.158171 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:07:28.158171 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:07:28.158171 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:07:28.158171 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:07:28.158171 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:07:28.158171 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:07:28.158171 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 04 09:07:28.158171 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:07:28.158171 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:07:28.158171 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:07:28.158171 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:28.158171 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:28.158171 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:28.158171 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:28.158171 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:28.159199 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:28.159199 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:28.159199 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:28.159199 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:28.159199 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:28.159199 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:28.159199 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:07:28.159199 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:07:28.159199 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:07:28.159199 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:07:28.159199 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:28.159199 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:28.159199 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:28.159199 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:28.159199 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:28.159199 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:28.160051 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:28.160051 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:28.160051 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:28.160051 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:28.160051 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:28.160051 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:28.160051 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:28.160051 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:28.160051 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:28.160051 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:28.160051 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:07:28.160051 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:07:28.160051 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 04 09:07:28.160051 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:07:28.164851 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 511/1005] 10.222.0.71 () {56 vars in 1105 bytes} [Mon Jul 4 09:07:27 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:28.166114 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135e3c31-46d7-40b0-90a4-92d8dbef496f None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:28.269930 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135e3c31-46d7-40b0-90a4-92d8dbef496f None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:28.304858 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f069eb-8270-46b0-b330-3f1458d72f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:28.305565 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f069eb-8270-46b0-b330-3f1458d72f72 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:28.305851 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f069eb-8270-46b0-b330-3f1458d72f72 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:28.306130 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f069eb-8270-46b0-b330-3f1458d72f72 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:28.306835 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f069eb-8270-46b0-b330-3f1458d72f72 None admin] RBAC: Adding query filter params (name=image) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:28.307143 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f069eb-8270-46b0-b330-3f1458d72f72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:28.308515 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f069eb-8270-46b0-b330-3f1458d72f72 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:28.309616 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:28.319553 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 495/1006] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:07:28 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:28.320598 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ead0d1-94c8-4029-89f4-74ebc3501c9a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:28.427467 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ead0d1-94c8-4029-89f4-74ebc3501c9a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:28.462244 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74836be8-57ce-4094-8307-fd7de24608a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:28.463185 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74836be8-57ce-4094-8307-fd7de24608a4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:28.463473 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74836be8-57ce-4094-8307-fd7de24608a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:28.463754 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74836be8-57ce-4094-8307-fd7de24608a4 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:28.464460 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74836be8-57ce-4094-8307-fd7de24608a4 None admin] RBAC: Adding query filter params (type=image) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:28.464859 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74836be8-57ce-4094-8307-fd7de24608a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:28.466219 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74836be8-57ce-4094-8307-fd7de24608a4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:28.471272 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 512/1007] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:07:28 2022] GET /identity/v3/services?type=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:28.706917 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9266057f-66dc-4bcb-911a-008d03368e52 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:28.707215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9266057f-66dc-4bcb-911a-008d03368e52 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:28.707493 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9266057f-66dc-4bcb-911a-008d03368e52 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:28.709007 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 496/1008] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:07:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:28.714811 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482bbb2-b0b0-40bf-98ef-62b2d36c27d6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:28.715183 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482bbb2-b0b0-40bf-98ef-62b2d36c27d6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:28.715542 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482bbb2-b0b0-40bf-98ef-62b2d36c27d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:28.727492 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:28.733382 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a482bbb2-b0b0-40bf-98ef-62b2d36c27d6 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:28.735272 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a482bbb2-b0b0-40bf-98ef-62b2d36c27d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:28.805071 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 513/1009] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:28 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:07:28.826471 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9266057f-66dc-4bcb-911a-008d03368e52 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:28.930602 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9266057f-66dc-4bcb-911a-008d03368e52 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:28.969390 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64a29c3-2bd7-4f73-b98d-72c36d3d528d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:28.970561 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a29c3-2bd7-4f73-b98d-72c36d3d528d None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:28.971057 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a29c3-2bd7-4f73-b98d-72c36d3d528d None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:28.971573 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a29c3-2bd7-4f73-b98d-72c36d3d528d None admin] PATH_INFO: `/v3/services/compute` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:28.972485 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64a29c3-2bd7-4f73-b98d-72c36d3d528d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:28.974161 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64a29c3-2bd7-4f73-b98d-72c36d3d528d None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:28.986711 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-b64a29c3-2bd7-4f73-b98d-72c36d3d528d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 04 09:07:28.986711 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:28.986711 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:07:28.986711 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:07:28.986711 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:07:28.986711 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:07:28.986711 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:07:28.986711 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:07:28.986711 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 04 09:07:28.986711 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:28.986711 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:07:28.986711 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:28.986711 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:28.986711 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:28.986711 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:28.986711 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:28.986711 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:28.987712 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:28.987712 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:28.987712 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:28.987712 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:28.987712 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:28.987712 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:28.987712 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:07:28.987712 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:07:28.987712 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:07:28.987712 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:07:28.987712 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:28.987712 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:28.987712 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:28.987712 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:28.987712 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:28.987712 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:28.988597 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:28.988597 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:28.988597 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:28.988597 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:28.988597 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:28.988597 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:28.988597 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:28.988597 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:28.988597 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:28.988597 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:28.988597 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:07:28.988597 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:07:28.988597 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 04 09:07:28.988597 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:28.990556 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 497/1010] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 4 09:07:28 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:28.996333 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a482bbb2-b0b0-40bf-98ef-62b2d36c27d6 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:29.097530 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a482bbb2-b0b0-40bf-98ef-62b2d36c27d6 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:29.132630 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e139fa87-b645-4c35-aded-78024301020f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:29.133321 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e139fa87-b645-4c35-aded-78024301020f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:29.133604 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e139fa87-b645-4c35-aded-78024301020f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:29.133883 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e139fa87-b645-4c35-aded-78024301020f None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:29.134579 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e139fa87-b645-4c35-aded-78024301020f None admin] RBAC: Adding query filter params (name=compute) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:29.134882 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e139fa87-b645-4c35-aded-78024301020f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:29.136235 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e139fa87-b645-4c35-aded-78024301020f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:29.138020 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:29.142263 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 514/1011] 10.222.0.71 () {56 vars in 1112 bytes} [Mon Jul 4 09:07:28 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:29.147871 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64a29c3-2bd7-4f73-b98d-72c36d3d528d None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:29.213935 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:29.255128 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64a29c3-2bd7-4f73-b98d-72c36d3d528d None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:29.289867 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f1a18a-9c4e-4dd5-9324-9138efc9d880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:29.290546 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f1a18a-9c4e-4dd5-9324-9138efc9d880 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:29.290833 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f1a18a-9c4e-4dd5-9324-9138efc9d880 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:29.291116 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f1a18a-9c4e-4dd5-9324-9138efc9d880 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:29.291885 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f1a18a-9c4e-4dd5-9324-9138efc9d880 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:29.292195 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f1a18a-9c4e-4dd5-9324-9138efc9d880 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:29.293640 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f1a18a-9c4e-4dd5-9324-9138efc9d880 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:29.303328 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 498/1012] 10.222.0.71 () {56 vars in 1112 bytes} [Mon Jul 4 09:07:29 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:07:29.305051 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e139fa87-b645-4c35-aded-78024301020f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:29.414017 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e139fa87-b645-4c35-aded-78024301020f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:29.449170 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a17d254-60a8-428c-93c8-9388d702346f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:29.449862 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a17d254-60a8-428c-93c8-9388d702346f None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:29.450153 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a17d254-60a8-428c-93c8-9388d702346f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:29.450436 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a17d254-60a8-428c-93c8-9388d702346f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:29.451182 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a17d254-60a8-428c-93c8-9388d702346f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:29.452549 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a17d254-60a8-428c-93c8-9388d702346f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:29.476271 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 515/1013] 10.222.0.71 () {60 vars in 1144 bytes} [Mon Jul 4 09:07:29 2022] POST /identity/v3/endpoints => generated 334 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:07:29.610094 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3f721-e30b-4c7b-9702-db4d106c00bc None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:29.610286 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3f721-e30b-4c7b-9702-db4d106c00bc None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:29.610714 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3f721-e30b-4c7b-9702-db4d106c00bc None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:29.612227 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 499/1014] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:07:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:29.618065 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411108d1-13f8-4717-bfc5-b1ca4dc72b22 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:29.618358 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411108d1-13f8-4717-bfc5-b1ca4dc72b22 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:29.618639 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411108d1-13f8-4717-bfc5-b1ca4dc72b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:29.636775 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-411108d1-13f8-4717-bfc5-b1ca4dc72b22 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:29.638025 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-411108d1-13f8-4717-bfc5-b1ca4dc72b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:29.690771 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:29.716840 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 516/1015] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:29 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:07:29.740360 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb3f721-e30b-4c7b-9702-db4d106c00bc None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:29.845616 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb3f721-e30b-4c7b-9702-db4d106c00bc None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:29.870785 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:29.880328 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70821cc-1b08-4458-b0da-1357b6ab35a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:29.881167 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70821cc-1b08-4458-b0da-1357b6ab35a1 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:29.881455 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70821cc-1b08-4458-b0da-1357b6ab35a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:29.882350 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70821cc-1b08-4458-b0da-1357b6ab35a1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:29.883681 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70821cc-1b08-4458-b0da-1357b6ab35a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:29.885153 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70821cc-1b08-4458-b0da-1357b6ab35a1 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:29.886948 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 500/1016] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:07:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:29.892634 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411108d1-13f8-4717-bfc5-b1ca4dc72b22 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:29.994743 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411108d1-13f8-4717-bfc5-b1ca4dc72b22 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:30.029140 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c434a5-3ace-4958-978e-cab18c9697bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:30.029800 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c434a5-3ace-4958-978e-cab18c9697bf None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:30.030082 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c434a5-3ace-4958-978e-cab18c9697bf None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:30.030360 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c434a5-3ace-4958-978e-cab18c9697bf None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:30.031110 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c434a5-3ace-4958-978e-cab18c9697bf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:30.032539 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c434a5-3ace-4958-978e-cab18c9697bf None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:30.065073 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 517/1017] 10.222.0.71 () {60 vars in 1141 bytes} [Mon Jul 4 09:07:29 2022] POST /identity/v3/projects => generated 311 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:07:30.511287 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9897a901-4da4-4291-a6dd-5948864bd246 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:30.511806 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9897a901-4da4-4291-a6dd-5948864bd246 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:30.511909 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9897a901-4da4-4291-a6dd-5948864bd246 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:30.513403 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 501/1018] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:07:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:30.518263 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f2b6a-07eb-4f4e-aed2-c143007bf23c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:30.518556 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f2b6a-07eb-4f4e-aed2-c143007bf23c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:30.518834 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f2b6a-07eb-4f4e-aed2-c143007bf23c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:30.536576 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-471f2b6a-07eb-4f4e-aed2-c143007bf23c None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:30.537779 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-471f2b6a-07eb-4f4e-aed2-c143007bf23c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:30.564622 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:30.609716 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 518/1019] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:30 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:07:30.632171 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9897a901-4da4-4291-a6dd-5948864bd246 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:30.736499 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9897a901-4da4-4291-a6dd-5948864bd246 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:30.769539 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c69029-fb6c-4879-b6e5-bbae22c154cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:30.770208 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c69029-fb6c-4879-b6e5-bbae22c154cd None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:30.770490 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c69029-fb6c-4879-b6e5-bbae22c154cd None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:30.770766 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c69029-fb6c-4879-b6e5-bbae22c154cd None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:30.771494 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c69029-fb6c-4879-b6e5-bbae22c154cd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:30.773523 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c69029-fb6c-4879-b6e5-bbae22c154cd None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:30.791471 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 502/1020] 10.222.0.71 () {60 vars in 1141 bytes} [Mon Jul 4 09:07:30 2022] POST /identity/v3/services => generated 242 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:07:31.659466 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3262eee2-e4f5-464d-a250-11be8116eb6c None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:31.659966 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3262eee2-e4f5-464d-a250-11be8116eb6c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:31.659966 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3262eee2-e4f5-464d-a250-11be8116eb6c None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:31.661393 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 519/1021] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:07:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:31.667042 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf13bb-ca32-4f11-9ca0-57145a6d4716 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:31.667432 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf13bb-ca32-4f11-9ca0-57145a6d4716 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:31.667619 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf13bb-ca32-4f11-9ca0-57145a6d4716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:31.684441 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-7faf13bb-ca32-4f11-9ca0-57145a6d4716 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:31.685666 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7faf13bb-ca32-4f11-9ca0-57145a6d4716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:31.734982 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:31.765081 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 503/1022] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:31 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:07:31.785544 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3262eee2-e4f5-464d-a250-11be8116eb6c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:31.893098 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3262eee2-e4f5-464d-a250-11be8116eb6c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:31.929350 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54614a9-aeaa-476e-acbd-9570e19bc3e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:31.930076 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54614a9-aeaa-476e-acbd-9570e19bc3e1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:31.930360 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54614a9-aeaa-476e-acbd-9570e19bc3e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:31.930635 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54614a9-aeaa-476e-acbd-9570e19bc3e1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:31.940893 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54614a9-aeaa-476e-acbd-9570e19bc3e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:07:31.949953 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-c54614a9-aeaa-476e-acbd-9570e19bc3e1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 09:07:31.949953 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:31.949953 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:31.949953 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:31.949953 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:31.949953 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:31.949953 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:31.949953 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:31.949953 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:31.949953 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:31.949953 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:31.949953 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:31.949953 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:07:31.949953 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:07:31.949953 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:07:31.949953 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:07:31.950896 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:07:31.950896 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:07:31.950896 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:07:31.950896 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:07:31.950896 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:31.950896 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:31.950896 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:31.950896 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:31.950896 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:31.950896 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:31.950896 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:31.950896 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:31.950896 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:31.950896 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:31.950896 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:31.950896 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:31.951764 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:31.951764 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:31.951764 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:31.951764 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:31.951764 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:07:31.951764 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:07:31.951764 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:07:31.951764 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:07:31.951764 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:07:31.951764 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:07:31.951764 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 09:07:31.951764 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:07:31.951764 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 520/1023] 10.222.0.71 () {56 vars in 1120 bytes} [Mon Jul 4 09:07:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:31.957376 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faf13bb-ca32-4f11-9ca0-57145a6d4716 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:32.059847 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faf13bb-ca32-4f11-9ca0-57145a6d4716 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:32.095537 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac656cc4-5107-4b94-a2b7-91ffdfca8a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:32.096208 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac656cc4-5107-4b94-a2b7-91ffdfca8a16 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:32.096565 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac656cc4-5107-4b94-a2b7-91ffdfca8a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:32.096852 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac656cc4-5107-4b94-a2b7-91ffdfca8a16 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:32.097626 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac656cc4-5107-4b94-a2b7-91ffdfca8a16 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:32.097938 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac656cc4-5107-4b94-a2b7-91ffdfca8a16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:32.099377 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac656cc4-5107-4b94-a2b7-91ffdfca8a16 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:32.109182 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 504/1024] 10.222.0.71 () {56 vars in 1115 bytes} [Mon Jul 4 09:07:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:32.114588 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54614a9-aeaa-476e-acbd-9570e19bc3e1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:32.141841 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:32.215834 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be74b36-c7ed-4e25-a151-9c89510326d5 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:32.216127 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be74b36-c7ed-4e25-a151-9c89510326d5 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:32.216415 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be74b36-c7ed-4e25-a151-9c89510326d5 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:32.217859 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 505/1025] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:07:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:07:32.220901 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54614a9-aeaa-476e-acbd-9570e19bc3e1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:32.222873 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db2fb2a-fd7e-41ee-854c-69fbd4e75f9e None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:32.223164 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db2fb2a-fd7e-41ee-854c-69fbd4e75f9e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:32.223447 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db2fb2a-fd7e-41ee-854c-69fbd4e75f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:32.246326 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-1db2fb2a-fd7e-41ee-854c-69fbd4e75f9e None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:32.247668 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-1db2fb2a-fd7e-41ee-854c-69fbd4e75f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:32.259214 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c396dfe8-1c6c-4021-8ffb-5ffc43e867da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:32.260302 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c396dfe8-1c6c-4021-8ffb-5ffc43e867da None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:32.260833 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c396dfe8-1c6c-4021-8ffb-5ffc43e867da None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:32.261392 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c396dfe8-1c6c-4021-8ffb-5ffc43e867da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:32.274331 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c396dfe8-1c6c-4021-8ffb-5ffc43e867da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:32.276042 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c396dfe8-1c6c-4021-8ffb-5ffc43e867da None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:32.285307 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:32.295881 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-c396dfe8-1c6c-4021-8ffb-5ffc43e867da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:07:32.295881 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:32.295881 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:32.295881 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:32.295881 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:32.295881 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:32.295881 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:32.295881 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:32.295881 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:32.295881 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:32.295881 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:32.295881 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:32.295881 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:07:32.295881 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:07:32.295881 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:07:32.296834 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:07:32.296834 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:32.296834 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:32.296834 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:32.296834 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:32.296834 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:32.296834 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:32.296834 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:32.296834 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:32.296834 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:32.296834 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:32.296834 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:32.296834 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:32.296834 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:32.296834 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:32.296834 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:32.296834 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:32.297794 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:07:32.297794 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:07:32.297794 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:07:32.297794 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:07:32.297794 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 521/1026] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:07:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:07:32.302730 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c396dfe8-1c6c-4021-8ffb-5ffc43e867da None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:32.326260 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 506/1027] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:32 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:07:32.352838 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db2fb2a-fd7e-41ee-854c-69fbd4e75f9e None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:32.392168 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:32.418047 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c396dfe8-1c6c-4021-8ffb-5ffc43e867da None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:32.455501 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b78b0f8-bae5-43fc-82af-087f0b828d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:32.456171 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b78b0f8-bae5-43fc-82af-087f0b828d4e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:32.456564 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b78b0f8-bae5-43fc-82af-087f0b828d4e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:32.456849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b78b0f8-bae5-43fc-82af-087f0b828d4e None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:32.457617 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b78b0f8-bae5-43fc-82af-087f0b828d4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:32.457923 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b78b0f8-bae5-43fc-82af-087f0b828d4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:32.459291 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b78b0f8-bae5-43fc-82af-087f0b828d4e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:32.460537 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db2fb2a-fd7e-41ee-854c-69fbd4e75f9e None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:32.477529 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 522/1028] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:07:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:07:32.478499 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b78b0f8-bae5-43fc-82af-087f0b828d4e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:32.499634 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdb5502-bf5c-4161-8827-3e4c2a53aa69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:32.500438 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb5502-bf5c-4161-8827-3e4c2a53aa69 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:32.500724 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb5502-bf5c-4161-8827-3e4c2a53aa69 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:32.501000 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb5502-bf5c-4161-8827-3e4c2a53aa69 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:32.502206 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdb5502-bf5c-4161-8827-3e4c2a53aa69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:32.503664 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdb5502-bf5c-4161-8827-3e4c2a53aa69 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:32.509072 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 507/1029] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:07:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:07:32.511941 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdb5502-bf5c-4161-8827-3e4c2a53aa69 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:32.554517 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:32.590319 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b78b0f8-bae5-43fc-82af-087f0b828d4e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:32.622206 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdb5502-bf5c-4161-8827-3e4c2a53aa69 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:32.633266 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8811a76c-be7e-4466-b3a4-6bdfb4451678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:32.634174 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8811a76c-be7e-4466-b3a4-6bdfb4451678 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:32.634454 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8811a76c-be7e-4466-b3a4-6bdfb4451678 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:32.634725 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8811a76c-be7e-4466-b3a4-6bdfb4451678 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:32.638167 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:32.643649 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8811a76c-be7e-4466-b3a4-6bdfb4451678 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:32.645531 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8811a76c-be7e-4466-b3a4-6bdfb4451678 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:32.657842 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c7ca2d-c989-451c-846e-15f1fc78fd72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:32.658515 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c7ca2d-c989-451c-846e-15f1fc78fd72 None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:32.658798 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c7ca2d-c989-451c-846e-15f1fc78fd72 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:32.659076 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c7ca2d-c989-451c-846e-15f1fc78fd72 None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:32.659801 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c7ca2d-c989-451c-846e-15f1fc78fd72 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:32.661256 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c7ca2d-c989-451c-846e-15f1fc78fd72 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:32.663563 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-8811a76c-be7e-4466-b3a4-6bdfb4451678 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 04 09:07:32.663563 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:32.663563 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:32.663563 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:32.663563 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:32.663563 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:32.663563 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:32.663563 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:32.663563 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:32.663563 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:32.663563 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:32.663563 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:32.663563 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:07:32.663563 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:07:32.663563 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:07:32.663563 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:07:32.664806 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:32.664806 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:32.664806 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:07:32.664806 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:07:32.664806 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:07:32.664806 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:07:32.664806 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:32.664806 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:32.664806 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:32.664806 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:32.664806 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:32.664806 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:32.664806 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:32.664806 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:32.664806 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:32.664806 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:32.664806 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:32.664806 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:32.666296 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:32.666296 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:32.666296 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:07:32.666296 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:07:32.666296 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:07:32.666296 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:07:32.666296 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:07:32.666296 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:07:32.666296 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 04 09:07:32.666296 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:07:32.669787 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 523/1030] 10.222.0.71 () {56 vars in 1093 bytes} [Mon Jul 4 09:07:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:07:32.671051 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8811a76c-be7e-4466-b3a4-6bdfb4451678 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:32.683368 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 508/1031] 10.222.0.71 () {60 vars in 1132 bytes} [Mon Jul 4 09:07:32 2022] POST /identity/v3/users => generated 285 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:07:32.786897 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8811a76c-be7e-4466-b3a4-6bdfb4451678 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:32.825854 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695a7d80-d38c-4997-9d9f-eb5a7cdd08a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:32.826543 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a7d80-d38c-4997-9d9f-eb5a7cdd08a7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:32.826831 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a7d80-d38c-4997-9d9f-eb5a7cdd08a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:32.827111 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a7d80-d38c-4997-9d9f-eb5a7cdd08a7 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:32.827999 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695a7d80-d38c-4997-9d9f-eb5a7cdd08a7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:32.828313 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695a7d80-d38c-4997-9d9f-eb5a7cdd08a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:32.829777 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695a7d80-d38c-4997-9d9f-eb5a7cdd08a7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:32.844852 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcac4d55-7826-47d0-8754-bd0d390c197c None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:32.845169 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcac4d55-7826-47d0-8754-bd0d390c197c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:32.845451 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcac4d55-7826-47d0-8754-bd0d390c197c None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:32.846820 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 509/1032] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:07:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:07:32.849392 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 524/1033] 10.222.0.71 () {56 vars in 1133 bytes} [Mon Jul 4 09:07:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:32.852567 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea27ef98-73aa-44af-b558-8823a655af87 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:32.852856 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea27ef98-73aa-44af-b558-8823a655af87 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:32.853190 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea27ef98-73aa-44af-b558-8823a655af87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:32.854885 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695a7d80-d38c-4997-9d9f-eb5a7cdd08a7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:32.873870 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-ea27ef98-73aa-44af-b558-8823a655af87 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:32.875080 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ea27ef98-73aa-44af-b558-8823a655af87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:32.920550 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:32.956599 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 510/1034] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:32 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:07:32.968560 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695a7d80-d38c-4997-9d9f-eb5a7cdd08a7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:32.979843 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea27ef98-73aa-44af-b558-8823a655af87 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:33.006464 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3710b3d1-37ce-4582-b7e9-e46734c9d7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:33.007808 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3710b3d1-37ce-4582-b7e9-e46734c9d7e8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:33.008110 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3710b3d1-37ce-4582-b7e9-e46734c9d7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:33.008391 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3710b3d1-37ce-4582-b7e9-e46734c9d7e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:33.009222 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:33.080737 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3710b3d1-37ce-4582-b7e9-e46734c9d7e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:33.082144 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3710b3d1-37ce-4582-b7e9-e46734c9d7e8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:33.088461 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea27ef98-73aa-44af-b558-8823a655af87 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:33.124247 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8907df63-5d45-459a-8f95-2239542e029d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:33.125075 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8907df63-5d45-459a-8f95-2239542e029d None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:33.125369 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8907df63-5d45-459a-8f95-2239542e029d None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:33.125659 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8907df63-5d45-459a-8f95-2239542e029d None admin] PATH_INFO: `/v3/services/image` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:33.126315 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8907df63-5d45-459a-8f95-2239542e029d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:33.127698 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8907df63-5d45-459a-8f95-2239542e029d None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:33.139686 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:33.140022 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-8907df63-5d45-459a-8f95-2239542e029d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 04 09:07:33.140022 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:33.140022 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:07:33.140022 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:07:33.140022 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:07:33.140022 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:07:33.140022 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:07:33.140022 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:07:33.140022 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 04 09:07:33.140022 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:33.140022 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:07:33.140022 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:33.140022 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:33.140022 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:33.140022 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:33.140022 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:33.140022 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:33.141135 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:33.141135 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:33.141135 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:33.141135 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:33.141135 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:33.141135 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:33.141135 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:07:33.141135 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:07:33.141135 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:07:33.141135 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:07:33.141135 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:33.141135 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:33.141135 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:33.141135 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:33.141135 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:33.141135 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:33.142145 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:33.142145 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:33.142145 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:33.142145 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:33.142145 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:33.142145 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:33.142145 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:33.142145 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:33.142145 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:33.142145 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:33.142145 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:07:33.142145 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:07:33.142145 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 04 09:07:33.142145 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:33.146314 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 511/1035] 10.222.0.71 () {56 vars in 1105 bytes} [Mon Jul 4 09:07:32 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:07:33.147191 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8907df63-5d45-459a-8f95-2239542e029d None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:33.196816 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 525/1036] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:07:32 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 04 09:07:33.197947 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3710b3d1-37ce-4582-b7e9-e46734c9d7e8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:33.244245 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:33.255404 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8907df63-5d45-459a-8f95-2239542e029d None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:33.290637 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cf2da9-ccf5-4b65-8636-decf953ad0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:33.291308 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cf2da9-ccf5-4b65-8636-decf953ad0d4 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:33.291597 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cf2da9-ccf5-4b65-8636-decf953ad0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:33.291879 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cf2da9-ccf5-4b65-8636-decf953ad0d4 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:33.292587 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cf2da9-ccf5-4b65-8636-decf953ad0d4 None admin] RBAC: Adding query filter params (name=image) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:33.292899 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cf2da9-ccf5-4b65-8636-decf953ad0d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:33.294304 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cf2da9-ccf5-4b65-8636-decf953ad0d4 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:33.299464 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 512/1037] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:07:33 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:33.305368 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cf2da9-ccf5-4b65-8636-decf953ad0d4 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:33.308576 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3710b3d1-37ce-4582-b7e9-e46734c9d7e8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:33.344637 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7defd514-ac28-4912-b8c2-f1ab8bb6964f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:33.345372 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7defd514-ac28-4912-b8c2-f1ab8bb6964f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:33.345652 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7defd514-ac28-4912-b8c2-f1ab8bb6964f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:33.345926 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7defd514-ac28-4912-b8c2-f1ab8bb6964f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:33.356776 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7defd514-ac28-4912-b8c2-f1ab8bb6964f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:33.358536 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7defd514-ac28-4912-b8c2-f1ab8bb6964f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:33.377524 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-7defd514-ac28-4912-b8c2-f1ab8bb6964f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:07:33.377524 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:33.377524 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:33.377524 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:33.377524 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:33.377524 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:33.377524 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:33.377524 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:33.377524 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:33.377524 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:33.377524 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:33.377524 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:33.377524 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:07:33.377524 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:07:33.377524 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:07:33.378508 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:07:33.378508 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:33.378508 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:33.378508 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:33.378508 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:33.378508 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:33.378508 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:33.378508 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:33.378508 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:33.378508 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:33.378508 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:33.378508 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:33.378508 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:33.378508 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:33.378508 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:33.378508 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:33.378508 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:33.379426 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:07:33.379426 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:07:33.379426 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:07:33.379426 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:07:33.379426 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 526/1038] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:07:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:33.385184 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7defd514-ac28-4912-b8c2-f1ab8bb6964f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:33.396016 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:33.415065 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cf2da9-ccf5-4b65-8636-decf953ad0d4 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:33.450931 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949be9c4-4da2-41b4-9fb6-f3adb88a8c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:33.451902 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949be9c4-4da2-41b4-9fb6-f3adb88a8c76 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:33.452188 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949be9c4-4da2-41b4-9fb6-f3adb88a8c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:33.452469 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949be9c4-4da2-41b4-9fb6-f3adb88a8c76 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:33.453218 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949be9c4-4da2-41b4-9fb6-f3adb88a8c76 None admin] RBAC: Adding query filter params (type=image) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:33.453524 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949be9c4-4da2-41b4-9fb6-f3adb88a8c76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:33.454887 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949be9c4-4da2-41b4-9fb6-f3adb88a8c76 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:33.460680 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 513/1039] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:07:33 2022] GET /identity/v3/services?type=image => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:07:33.466313 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949be9c4-4da2-41b4-9fb6-f3adb88a8c76 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:33.486981 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:33.501666 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7defd514-ac28-4912-b8c2-f1ab8bb6964f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:33.536844 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9597c47d-fedb-4dab-b898-f7fc37e112af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:33.537564 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9597c47d-fedb-4dab-b898-f7fc37e112af None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:33.537910 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9597c47d-fedb-4dab-b898-f7fc37e112af None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:33.538730 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9597c47d-fedb-4dab-b898-f7fc37e112af None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:33.539453 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9597c47d-fedb-4dab-b898-f7fc37e112af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:33.539761 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9597c47d-fedb-4dab-b898-f7fc37e112af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:33.541148 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9597c47d-fedb-4dab-b898-f7fc37e112af None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:33.552741 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 527/1040] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:07:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:33.558691 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9597c47d-fedb-4dab-b898-f7fc37e112af None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:33.578184 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949be9c4-4da2-41b4-9fb6-f3adb88a8c76 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:33.613970 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77e839c-443d-4d7f-8f21-fe850f5349f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:33.614611 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77e839c-443d-4d7f-8f21-fe850f5349f1 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:33.614901 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77e839c-443d-4d7f-8f21-fe850f5349f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:33.615588 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77e839c-443d-4d7f-8f21-fe850f5349f1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:33.617206 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77e839c-443d-4d7f-8f21-fe850f5349f1 None admin] RBAC: Adding query filter params (interface=public, service_id=e0ee44b4193343fabd74ba562a8a5322, region_id=RegionOne) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:33.617443 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77e839c-443d-4d7f-8f21-fe850f5349f1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:33.618817 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77e839c-443d-4d7f-8f21-fe850f5349f1 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:33.624171 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 514/1041] 10.222.0.71 () {56 vars in 1251 bytes} [Mon Jul 4 09:07:33 2022] GET /identity/v3/endpoints?service_id=e0ee44b4193343fabd74ba562a8a5322&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:07:33.629358 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:33.629964 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77e839c-443d-4d7f-8f21-fe850f5349f1 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:33.670174 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9597c47d-fedb-4dab-b898-f7fc37e112af None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:33.706578 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b332085-5a8c-4db9-a013-bc1988806632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:33.707497 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b332085-5a8c-4db9-a013-bc1988806632 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:33.707785 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b332085-5a8c-4db9-a013-bc1988806632 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:33.708066 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b332085-5a8c-4db9-a013-bc1988806632 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:33.715069 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:33.717067 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b332085-5a8c-4db9-a013-bc1988806632 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:33.718720 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b332085-5a8c-4db9-a013-bc1988806632 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:33.732950 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-5b332085-5a8c-4db9-a013-bc1988806632 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:07:33.732950 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:33.732950 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:33.732950 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:33.732950 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:33.732950 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:33.732950 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:33.732950 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:33.732950 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:33.732950 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:33.732950 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:33.732950 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:33.732950 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:07:33.732950 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:07:33.732950 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:07:33.734094 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:07:33.734094 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:33.734094 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:33.734094 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:33.734094 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:33.734094 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:33.734094 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:33.734094 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:33.734094 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:33.734094 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:33.734094 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:33.734094 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:33.734094 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:33.734094 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:33.734094 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:33.734094 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:33.734094 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:33.735018 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:07:33.735018 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:07:33.735018 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:07:33.735018 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:07:33.735018 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:07:33.735018 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:07:33.735018 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:07:33.735018 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:07:33.735018 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 528/1042] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 4 09:07:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:33.738100 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77e839c-443d-4d7f-8f21-fe850f5349f1 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:33.740266 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b332085-5a8c-4db9-a013-bc1988806632 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:33.776318 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa13712e-1da3-44d8-9796-709a2ccd7ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:33.777698 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13712e-1da3-44d8-9796-709a2ccd7ee3 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:33.777967 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13712e-1da3-44d8-9796-709a2ccd7ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:33.778256 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13712e-1da3-44d8-9796-709a2ccd7ee3 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:33.778953 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa13712e-1da3-44d8-9796-709a2ccd7ee3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:33.780331 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa13712e-1da3-44d8-9796-709a2ccd7ee3 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:33.797075 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 515/1043] 10.222.0.71 () {56 vars in 1087 bytes} [Mon Jul 4 09:07:33 2022] GET /identity/v3/services => generated 1957 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:07:33.873902 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b332085-5a8c-4db9-a013-bc1988806632 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:33.902902 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:33.914746 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf047779-84f8-4903-a3a1-34a9fd648e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:33.915437 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf047779-84f8-4903-a3a1-34a9fd648e53 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:33.915725 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf047779-84f8-4903-a3a1-34a9fd648e53 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:33.916008 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf047779-84f8-4903-a3a1-34a9fd648e53 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:33.916816 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf047779-84f8-4903-a3a1-34a9fd648e53 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:33.917163 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf047779-84f8-4903-a3a1-34a9fd648e53 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:33.918615 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf047779-84f8-4903-a3a1-34a9fd648e53 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:33.932526 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 529/1044] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 4 09:07:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:33.939182 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa13712e-1da3-44d8-9796-709a2ccd7ee3 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:34.047419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa13712e-1da3-44d8-9796-709a2ccd7ee3 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:34.082207 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c979f3-f6c9-499c-8bbc-ae27fb7efa74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:34.082864 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c979f3-f6c9-499c-8bbc-ae27fb7efa74 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:34.083145 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c979f3-f6c9-499c-8bbc-ae27fb7efa74 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:34.083427 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c979f3-f6c9-499c-8bbc-ae27fb7efa74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:34.084248 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c979f3-f6c9-499c-8bbc-ae27fb7efa74 None admin] RBAC: Adding query filter params (role.id=86b73ebb032e430c9c08b78a01898cd7, scope.project.id=4076e316243946e980ce47386b18b28f, user.id=15039da209ec445d93d49e2bd6d854eb) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:34.084560 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c979f3-f6c9-499c-8bbc-ae27fb7efa74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:34.086426 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c979f3-f6c9-499c-8bbc-ae27fb7efa74 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:34.097183 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 516/1045] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 4 09:07:33 2022] GET /identity/v3/role_assignments?user.id=15039da209ec445d93d49e2bd6d854eb&scope.project.id=4076e316243946e980ce47386b18b28f&role.id=86b73ebb032e430c9c08b78a01898cd7 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:34.801896 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915a1d3-d2bf-4492-afba-96e3ecedda33 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:34.802455 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915a1d3-d2bf-4492-afba-96e3ecedda33 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:34.802455 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915a1d3-d2bf-4492-afba-96e3ecedda33 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:34.804632 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 530/1046] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:07:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:34.809648 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afe280e-7274-45d9-a1d3-97fc02a47229 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:34.810070 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afe280e-7274-45d9-a1d3-97fc02a47229 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:34.810253 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afe280e-7274-45d9-a1d3-97fc02a47229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:34.827517 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-6afe280e-7274-45d9-a1d3-97fc02a47229 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:34.830155 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-6afe280e-7274-45d9-a1d3-97fc02a47229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:34.833860 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:34.900549 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 517/1047] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:34 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:07:34.923660 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b915a1d3-d2bf-4492-afba-96e3ecedda33 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:35.032205 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b915a1d3-d2bf-4492-afba-96e3ecedda33 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:35.065938 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89520ec3-bb04-4345-bd47-84a7405b84e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:35.066740 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89520ec3-bb04-4345-bd47-84a7405b84e7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:35.066983 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89520ec3-bb04-4345-bd47-84a7405b84e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:35.067267 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89520ec3-bb04-4345-bd47-84a7405b84e7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:35.078422 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89520ec3-bb04-4345-bd47-84a7405b84e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:07:35.086290 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-89520ec3-bb04-4345-bd47-84a7405b84e7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:07:35.086290 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:35.086290 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:35.086290 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:35.086290 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:35.086290 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:35.086290 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:35.086290 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:35.086290 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:35.086290 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:35.086290 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:35.086290 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:35.086290 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:07:35.086290 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:07:35.086290 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:07:35.086290 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:07:35.087278 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:07:35.087278 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:07:35.087278 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:07:35.087278 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:07:35.087278 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:35.087278 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:35.087278 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:35.087278 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:35.087278 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:35.087278 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:35.087278 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:35.087278 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:35.087278 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:35.087278 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:35.087278 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:35.087278 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:35.088409 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:35.088409 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:35.088409 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:35.088409 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:35.088409 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:07:35.088409 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:07:35.088409 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:07:35.088409 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:07:35.088409 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:07:35.088409 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:07:35.088409 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:07:35.088409 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:07:35.092786 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 531/1048] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 4 09:07:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:35.093637 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afe280e-7274-45d9-a1d3-97fc02a47229 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:35.193989 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afe280e-7274-45d9-a1d3-97fc02a47229 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:35.228962 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4df8a-8642-4c6f-ad01-552919bb9ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:35.229673 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4df8a-8642-4c6f-ad01-552919bb9ad0 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:35.229948 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4df8a-8642-4c6f-ad01-552919bb9ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:35.230229 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4df8a-8642-4c6f-ad01-552919bb9ad0 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:35.230984 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c4df8a-8642-4c6f-ad01-552919bb9ad0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:35.231291 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c4df8a-8642-4c6f-ad01-552919bb9ad0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:35.232679 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c4df8a-8642-4c6f-ad01-552919bb9ad0 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:35.242393 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 518/1049] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:07:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:35.247850 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89520ec3-bb04-4345-bd47-84a7405b84e7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:35.300635 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:35.356907 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89520ec3-bb04-4345-bd47-84a7405b84e7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:35.394802 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef0c7c1-fde2-4555-b54c-af138bcac062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:35.395742 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef0c7c1-fde2-4555-b54c-af138bcac062 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:35.396054 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef0c7c1-fde2-4555-b54c-af138bcac062 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:35.396337 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef0c7c1-fde2-4555-b54c-af138bcac062 None admin] PATH_INFO: `/v3/users/cdb7ee5dcac34293b8fd0af0f7ee0ebb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:35.411104 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef0c7c1-fde2-4555-b54c-af138bcac062 None admin] RBAC: Authorizing `identity:get_user(user_id=cdb7ee5dcac34293b8fd0af0f7ee0ebb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:35.412545 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef0c7c1-fde2-4555-b54c-af138bcac062 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:35.415167 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 532/1050] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 4 09:07:35 2022] GET /identity/v3/users/cdb7ee5dcac34293b8fd0af0f7ee0ebb => generated 285 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:07:35.420598 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4df8a-8642-4c6f-ad01-552919bb9ad0 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:35.423613 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:35.525223 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4df8a-8642-4c6f-ad01-552919bb9ad0 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:35.560503 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4556ce7-7c1e-40f9-bc3e-4e850dde15e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:35.561187 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4556ce7-7c1e-40f9-bc3e-4e850dde15e1 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:35.561475 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4556ce7-7c1e-40f9-bc3e-4e850dde15e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:35.561754 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4556ce7-7c1e-40f9-bc3e-4e850dde15e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:35.639638 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4556ce7-7c1e-40f9-bc3e-4e850dde15e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:35.641006 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4556ce7-7c1e-40f9-bc3e-4e850dde15e1 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:35.642894 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:35.745765 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 519/1051] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:07:35 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:07:35.751585 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef0c7c1-fde2-4555-b54c-af138bcac062 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:35.853651 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef0c7c1-fde2-4555-b54c-af138bcac062 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:35.856028 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:35.894526 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01923ac-43bb-4572-8a0a-801a2e7fe3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:35.895304 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01923ac-43bb-4572-8a0a-801a2e7fe3df None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:35.895588 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01923ac-43bb-4572-8a0a-801a2e7fe3df None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:35.895865 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01923ac-43bb-4572-8a0a-801a2e7fe3df None admin] PATH_INFO: `/v3/projects/97293ed76327477d86aac1c047ff78d5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:35.897078 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01923ac-43bb-4572-8a0a-801a2e7fe3df None admin] RBAC: Authorizing `identity:get_project(project_id=97293ed76327477d86aac1c047ff78d5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:35.899149 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01923ac-43bb-4572-8a0a-801a2e7fe3df None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:35.901003 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 533/1052] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 4 09:07:35 2022] GET /identity/v3/projects/97293ed76327477d86aac1c047ff78d5 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:35.904905 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c78dcab-c009-4ec7-9c47-ec9618002a2d None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:35.905529 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c78dcab-c009-4ec7-9c47-ec9618002a2d None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:35.905529 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c78dcab-c009-4ec7-9c47-ec9618002a2d None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:35.906874 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01923ac-43bb-4572-8a0a-801a2e7fe3df None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:35.907336 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 520/1053] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:07:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:35.911890 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02febb7f-bafe-4cbc-9624-9ffd2483e811 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:35.912181 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02febb7f-bafe-4cbc-9624-9ffd2483e811 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:35.912454 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02febb7f-bafe-4cbc-9624-9ffd2483e811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:35.930816 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-02febb7f-bafe-4cbc-9624-9ffd2483e811 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:35.932167 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-02febb7f-bafe-4cbc-9624-9ffd2483e811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:36.008997 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 521/1054] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:35 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:07:36.019785 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01923ac-43bb-4572-8a0a-801a2e7fe3df None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:36.032422 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02febb7f-bafe-4cbc-9624-9ffd2483e811 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:36.057293 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76296a5a-76a4-404b-83ef-b3404014f5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:36.057952 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76296a5a-76a4-404b-83ef-b3404014f5e1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:36.058255 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76296a5a-76a4-404b-83ef-b3404014f5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:36.058522 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76296a5a-76a4-404b-83ef-b3404014f5e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:36.059486 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76296a5a-76a4-404b-83ef-b3404014f5e1 None admin] RBAC: Adding query filter params (role.id=f36e053b442540f08041ac7878760f65, scope.project.id=97293ed76327477d86aac1c047ff78d5, user.id=cdb7ee5dcac34293b8fd0af0f7ee0ebb) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:36.059806 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76296a5a-76a4-404b-83ef-b3404014f5e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:36.061265 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76296a5a-76a4-404b-83ef-b3404014f5e1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:36.065618 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:36.073264 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 534/1055] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 4 09:07:35 2022] GET /identity/v3/role_assignments?user.id=cdb7ee5dcac34293b8fd0af0f7ee0ebb&scope.project.id=97293ed76327477d86aac1c047ff78d5&role.id=f36e053b442540f08041ac7878760f65 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:07:36.105687 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b9c78-0bcb-415a-826e-1adc79cf3950 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:36.106349 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b9c78-0bcb-415a-826e-1adc79cf3950 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:36.106349 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b9c78-0bcb-415a-826e-1adc79cf3950 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:36.107764 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 535/1056] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:07:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:36.114245 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7392113-9e72-4187-b66f-a67bc035b1f5 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:36.114864 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7392113-9e72-4187-b66f-a67bc035b1f5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:36.115206 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7392113-9e72-4187-b66f-a67bc035b1f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:36.136664 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c7392113-9e72-4187-b66f-a67bc035b1f5 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:36.138732 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c7392113-9e72-4187-b66f-a67bc035b1f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:36.150472 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02febb7f-bafe-4cbc-9624-9ffd2483e811 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:36.175106 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:36.188842 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c9122-4504-4f55-ac14-4eeb00de2a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:36.189705 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c9122-4504-4f55-ac14-4eeb00de2a10 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:36.189993 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c9122-4504-4f55-ac14-4eeb00de2a10 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:36.190273 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c9122-4504-4f55-ac14-4eeb00de2a10 None admin] PATH_INFO: `/v3/services/image` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:36.190945 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805c9122-4504-4f55-ac14-4eeb00de2a10 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:36.192351 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805c9122-4504-4f55-ac14-4eeb00de2a10 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:36.212767 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-805c9122-4504-4f55-ac14-4eeb00de2a10 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 04 09:07:36.212767 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:36.212767 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:07:36.212767 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:07:36.212767 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:07:36.212767 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:07:36.212767 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:07:36.212767 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:07:36.212767 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 04 09:07:36.212767 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:36.212767 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:07:36.212767 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:36.212767 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:36.212767 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:36.212767 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:36.212767 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:36.212767 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:36.214068 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:36.214068 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:36.214068 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:36.214068 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:36.214068 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:36.214068 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:36.214068 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:07:36.214068 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:07:36.214068 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:07:36.214068 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:07:36.214068 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:36.214068 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:36.214068 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:36.214068 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:36.214068 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:36.214068 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:36.215097 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:36.215097 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:36.215097 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:36.215097 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:36.215097 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:36.215097 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:36.215097 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:36.215097 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:36.215097 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:36.215097 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:36.215097 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:07:36.215097 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:07:36.215097 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 04 09:07:36.215097 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:36.215097 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 522/1057] 10.222.0.71 () {56 vars in 1105 bytes} [Mon Jul 4 09:07:36 2022] GET /identity/v3/services/image => generated 86 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:07:36.222777 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c9122-4504-4f55-ac14-4eeb00de2a10 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:36.225756 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 536/1058] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:36 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:07:36.250021 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7392113-9e72-4187-b66f-a67bc035b1f5 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:36.328349 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:36.336813 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c9122-4504-4f55-ac14-4eeb00de2a10 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:36.356567 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7392113-9e72-4187-b66f-a67bc035b1f5 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:36.372585 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32710f3-9427-428e-b49f-a3e9f83d9913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:36.373313 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32710f3-9427-428e-b49f-a3e9f83d9913 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:36.373649 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32710f3-9427-428e-b49f-a3e9f83d9913 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:36.373877 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32710f3-9427-428e-b49f-a3e9f83d9913 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:36.374584 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32710f3-9427-428e-b49f-a3e9f83d9913 None admin] RBAC: Adding query filter params (name=image) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:36.374890 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32710f3-9427-428e-b49f-a3e9f83d9913 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:36.376265 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32710f3-9427-428e-b49f-a3e9f83d9913 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:36.386336 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 523/1059] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:07:36 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:07:36.387959 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32710f3-9427-428e-b49f-a3e9f83d9913 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:36.403768 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c74c3-81d5-4a5b-98f4-9bd82fa3ead2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:36.404605 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c74c3-81d5-4a5b-98f4-9bd82fa3ead2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:36.404897 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c74c3-81d5-4a5b-98f4-9bd82fa3ead2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:36.405227 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c74c3-81d5-4a5b-98f4-9bd82fa3ead2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:36.419254 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3c74c3-81d5-4a5b-98f4-9bd82fa3ead2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:07:36.427751 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-0b3c74c3-81d5-4a5b-98f4-9bd82fa3ead2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 09:07:36.427751 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:36.427751 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:36.427751 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:36.427751 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:36.427751 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:36.427751 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:36.427751 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:36.427751 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:36.427751 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:36.427751 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:36.427751 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:36.427751 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:07:36.427751 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:07:36.427751 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:07:36.427751 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:07:36.428884 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:07:36.428884 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:07:36.428884 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:07:36.428884 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:07:36.428884 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:36.428884 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:36.428884 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:36.428884 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:36.428884 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:36.428884 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:36.428884 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:36.428884 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:36.428884 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:36.428884 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:36.428884 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:36.428884 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:36.434539 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:36.434539 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:36.434539 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:36.434539 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:36.434539 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:07:36.434539 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:07:36.434539 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:07:36.434539 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:07:36.434539 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:07:36.434539 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:07:36.434539 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 09:07:36.434539 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:07:36.434539 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 537/1060] 10.222.0.71 () {56 vars in 1120 bytes} [Mon Jul 4 09:07:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:07:36.436045 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c74c3-81d5-4a5b-98f4-9bd82fa3ead2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:36.469403 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:36.500343 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32710f3-9427-428e-b49f-a3e9f83d9913 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:36.541662 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db36a42a-3463-4160-b2f9-b5108f4ed386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:36.542316 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db36a42a-3463-4160-b2f9-b5108f4ed386 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:36.542601 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db36a42a-3463-4160-b2f9-b5108f4ed386 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:36.542879 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db36a42a-3463-4160-b2f9-b5108f4ed386 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:36.543587 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db36a42a-3463-4160-b2f9-b5108f4ed386 None admin] RBAC: Adding query filter params (type=image) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:36.543896 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db36a42a-3463-4160-b2f9-b5108f4ed386 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:36.545280 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db36a42a-3463-4160-b2f9-b5108f4ed386 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:36.548066 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c74c3-81d5-4a5b-98f4-9bd82fa3ead2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:36.550410 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 524/1061] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:07:36 2022] GET /identity/v3/services?type=image => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:07:36.557259 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db36a42a-3463-4160-b2f9-b5108f4ed386 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:36.584263 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:36.592048 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c46c58-1874-4b42-9033-d900f2b9edd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:36.592705 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c46c58-1874-4b42-9033-d900f2b9edd5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:36.592990 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c46c58-1874-4b42-9033-d900f2b9edd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:36.593299 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c46c58-1874-4b42-9033-d900f2b9edd5 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:36.594049 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c46c58-1874-4b42-9033-d900f2b9edd5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:36.594386 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c46c58-1874-4b42-9033-d900f2b9edd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:36.595726 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c46c58-1874-4b42-9033-d900f2b9edd5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:36.606036 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 538/1062] 10.222.0.71 () {56 vars in 1115 bytes} [Mon Jul 4 09:07:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:36.612328 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c46c58-1874-4b42-9033-d900f2b9edd5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:36.670110 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db36a42a-3463-4160-b2f9-b5108f4ed386 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:36.706552 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6c0b9f-d792-4088-aa9a-1dd56d1172ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:36.707222 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c0b9f-d792-4088-aa9a-1dd56d1172ac None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:36.707511 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c0b9f-d792-4088-aa9a-1dd56d1172ac None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:36.707786 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c0b9f-d792-4088-aa9a-1dd56d1172ac None admin] PATH_INFO: `/v3/endpoints` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:36.708531 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6c0b9f-d792-4088-aa9a-1dd56d1172ac None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:36.709962 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6c0b9f-d792-4088-aa9a-1dd56d1172ac None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:36.711949 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:36.727106 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c46c58-1874-4b42-9033-d900f2b9edd5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:36.730264 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 525/1063] 10.222.0.71 () {60 vars in 1144 bytes} [Mon Jul 4 09:07:36 2022] POST /identity/v3/endpoints => generated 327 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:07:36.781237 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0b8fd3-1d89-4814-a085-c775f6f96197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:36.782158 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0b8fd3-1d89-4814-a085-c775f6f96197 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:36.782559 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0b8fd3-1d89-4814-a085-c775f6f96197 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:36.782849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0b8fd3-1d89-4814-a085-c775f6f96197 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:36.795507 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0b8fd3-1d89-4814-a085-c775f6f96197 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:36.796990 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0b8fd3-1d89-4814-a085-c775f6f96197 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:36.816497 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-5f0b8fd3-1d89-4814-a085-c775f6f96197 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:07:36.816497 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:36.816497 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:36.816497 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:36.816497 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:36.816497 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:36.816497 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:36.816497 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:36.816497 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:36.816497 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:36.816497 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:36.816497 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:36.816497 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:07:36.816497 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:07:36.816497 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:07:36.823888 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:07:36.823888 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:36.823888 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:36.823888 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:36.823888 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:36.823888 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:36.823888 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:36.823888 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:36.823888 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:36.823888 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:36.823888 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:36.823888 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:36.823888 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:36.823888 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:36.823888 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:36.823888 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:36.823888 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:36.825201 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:07:36.825201 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:07:36.825201 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:07:36.825201 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:07:36.825201 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 539/1064] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:07:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:07:36.826296 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6c0b9f-d792-4088-aa9a-1dd56d1172ac None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:36.916059 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:36.940322 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6c0b9f-d792-4088-aa9a-1dd56d1172ac None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:36.974910 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e00ffd-c23c-4e2b-a5dc-39a4d2c1671a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:36.976209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e00ffd-c23c-4e2b-a5dc-39a4d2c1671a None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:36.976581 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e00ffd-c23c-4e2b-a5dc-39a4d2c1671a None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:36.976880 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e00ffd-c23c-4e2b-a5dc-39a4d2c1671a None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:36.977635 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e00ffd-c23c-4e2b-a5dc-39a4d2c1671a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:36.977946 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e00ffd-c23c-4e2b-a5dc-39a4d2c1671a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:36.979312 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e00ffd-c23c-4e2b-a5dc-39a4d2c1671a None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:36.992237 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 526/1065] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:07:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:36.997644 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0b8fd3-1d89-4814-a085-c775f6f96197 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:37.099440 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0b8fd3-1d89-4814-a085-c775f6f96197 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:37.134659 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c848418-c924-4283-9f36-338adebc9f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:37.135458 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c848418-c924-4283-9f36-338adebc9f4a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:37.135742 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c848418-c924-4283-9f36-338adebc9f4a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:37.136047 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c848418-c924-4283-9f36-338adebc9f4a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:37.138796 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:37.145128 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c848418-c924-4283-9f36-338adebc9f4a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:37.146561 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c848418-c924-4283-9f36-338adebc9f4a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:37.160559 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-4c848418-c924-4283-9f36-338adebc9f4a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 04 09:07:37.160559 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:37.160559 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:37.160559 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:37.160559 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:37.160559 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:37.160559 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:37.160559 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:37.160559 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:37.160559 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:37.160559 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:37.160559 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:37.160559 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:07:37.160559 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:07:37.160559 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:07:37.160559 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:07:37.161700 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:37.161700 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:37.161700 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:07:37.161700 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:07:37.161700 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:07:37.161700 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:07:37.161700 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:37.161700 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:37.161700 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:37.161700 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:37.161700 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:37.161700 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:37.161700 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:37.161700 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:37.161700 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:37.161700 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:37.161700 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:37.161700 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:37.162694 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:37.162694 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:37.162694 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:07:37.162694 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:07:37.162694 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:07:37.162694 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:07:37.162694 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:07:37.162694 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:07:37.162694 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 04 09:07:37.162694 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:07:37.165127 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 540/1066] 10.222.0.71 () {56 vars in 1093 bytes} [Mon Jul 4 09:07:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:37.168034 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e00ffd-c23c-4e2b-a5dc-39a4d2c1671a None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:37.278264 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e00ffd-c23c-4e2b-a5dc-39a4d2c1671a None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:37.312391 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c19957e-f745-42cb-8511-d1babba5f679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:37.313086 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19957e-f745-42cb-8511-d1babba5f679 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:37.313378 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19957e-f745-42cb-8511-d1babba5f679 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:37.313657 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19957e-f745-42cb-8511-d1babba5f679 None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:37.314503 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c19957e-f745-42cb-8511-d1babba5f679 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:37.314816 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c19957e-f745-42cb-8511-d1babba5f679 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:37.316249 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c19957e-f745-42cb-8511-d1babba5f679 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:37.325612 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:37.335999 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 527/1067] 10.222.0.71 () {56 vars in 1133 bytes} [Mon Jul 4 09:07:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:37.341800 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c848418-c924-4283-9f36-338adebc9f4a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:37.451781 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c848418-c924-4283-9f36-338adebc9f4a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:37.486027 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89b7d65-594d-4044-915c-d3909584c5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:37.486821 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b7d65-594d-4044-915c-d3909584c5ca None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:37.487103 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b7d65-594d-4044-915c-d3909584c5ca None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:37.487376 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b7d65-594d-4044-915c-d3909584c5ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:37.498538 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89b7d65-594d-4044-915c-d3909584c5ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:37.500044 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89b7d65-594d-4044-915c-d3909584c5ca None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:37.517935 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-a89b7d65-594d-4044-915c-d3909584c5ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:07:37.517935 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:37.517935 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:37.517935 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:37.517935 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:37.517935 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:37.517935 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:37.517935 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:37.517935 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:37.517935 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:37.517935 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:37.517935 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:37.517935 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:07:37.517935 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:07:37.517935 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:07:37.518940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:07:37.518940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:37.518940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:37.518940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:37.518940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:37.518940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:37.518940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:37.518940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:37.518940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:37.518940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:37.518940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:37.518940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:37.518940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:37.518940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:37.518940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:37.518940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:37.518940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:37.524867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:07:37.524867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:07:37.524867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:07:37.524867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:07:37.524867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 541/1068] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:07:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:07:37.526700 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c19957e-f745-42cb-8511-d1babba5f679 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:37.633217 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c19957e-f745-42cb-8511-d1babba5f679 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:37.668279 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cda5c04-0657-441f-8c15-d5d7508b2b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:37.668946 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda5c04-0657-441f-8c15-d5d7508b2b69 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:37.669258 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda5c04-0657-441f-8c15-d5d7508b2b69 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:37.669540 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda5c04-0657-441f-8c15-d5d7508b2b69 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:37.670257 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cda5c04-0657-441f-8c15-d5d7508b2b69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:37.670566 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cda5c04-0657-441f-8c15-d5d7508b2b69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:37.671932 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cda5c04-0657-441f-8c15-d5d7508b2b69 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:37.688793 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 528/1069] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:07:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:07:37.690409 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89b7d65-594d-4044-915c-d3909584c5ca None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:37.742727 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:37.796049 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89b7d65-594d-4044-915c-d3909584c5ca None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:37.829905 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36422d2f-355a-4e18-9167-6d1aca07cf7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:37.830704 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36422d2f-355a-4e18-9167-6d1aca07cf7e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:37.830990 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36422d2f-355a-4e18-9167-6d1aca07cf7e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:37.831272 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36422d2f-355a-4e18-9167-6d1aca07cf7e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:37.838457 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36422d2f-355a-4e18-9167-6d1aca07cf7e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:37.839910 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36422d2f-355a-4e18-9167-6d1aca07cf7e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:37.858312 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-36422d2f-355a-4e18-9167-6d1aca07cf7e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:07:37.858312 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:37.858312 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:37.858312 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:37.858312 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:37.858312 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:37.858312 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:37.858312 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:37.858312 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:37.858312 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:37.858312 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:37.858312 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:37.858312 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:07:37.858312 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:07:37.858312 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:07:37.859261 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:07:37.859261 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:37.859261 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:37.859261 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:37.859261 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:37.859261 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:37.859261 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:37.859261 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:37.859261 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:37.859261 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:37.859261 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:37.859261 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:37.859261 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:37.859261 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:37.859261 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:37.859261 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:37.859261 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:37.860183 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:07:37.860183 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:07:37.860183 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:07:37.860183 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:07:37.860183 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:07:37.860183 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:07:37.860183 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:07:37.860183 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:07:37.860183 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 542/1070] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 4 09:07:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:37.865585 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cda5c04-0657-441f-8c15-d5d7508b2b69 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:37.928795 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:37.972314 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cda5c04-0657-441f-8c15-d5d7508b2b69 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:38.005411 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e9c3f-ddfa-44e0-bc23-9d8542af1e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:38.006239 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e9c3f-ddfa-44e0-bc23-9d8542af1e97 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:38.007140 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e9c3f-ddfa-44e0-bc23-9d8542af1e97 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:38.007537 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e9c3f-ddfa-44e0-bc23-9d8542af1e97 None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:38.008330 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695e9c3f-ddfa-44e0-bc23-9d8542af1e97 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:38.008642 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695e9c3f-ddfa-44e0-bc23-9d8542af1e97 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:38.010202 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695e9c3f-ddfa-44e0-bc23-9d8542af1e97 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:38.023916 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 529/1071] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 4 09:07:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:38.029538 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36422d2f-355a-4e18-9167-6d1aca07cf7e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:38.130883 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36422d2f-355a-4e18-9167-6d1aca07cf7e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:38.141596 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:38.157942 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98da6c8-d073-422f-93c0-5158cafea0a2 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:38.158317 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98da6c8-d073-422f-93c0-5158cafea0a2 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:38.158599 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98da6c8-d073-422f-93c0-5158cafea0a2 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:38.160212 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 530/1072] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:07:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:07:38.166139 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751b3eb-2618-4d24-84dd-e06432fd6e1c None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:38.166451 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751b3eb-2618-4d24-84dd-e06432fd6e1c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:38.166705 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751b3eb-2618-4d24-84dd-e06432fd6e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:38.172076 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f340b-63b8-4ffd-b83b-3dface3f0f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:38.173729 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f340b-63b8-4ffd-b83b-3dface3f0f95 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:38.174168 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f340b-63b8-4ffd-b83b-3dface3f0f95 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:38.174450 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f340b-63b8-4ffd-b83b-3dface3f0f95 None admin] PATH_INFO: `/v3/projects/4076e316243946e980ce47386b18b28f/users/15039da209ec445d93d49e2bd6d854eb/roles/86b73ebb032e430c9c08b78a01898cd7` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:38.176729 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148f340b-63b8-4ffd-b83b-3dface3f0f95 None admin] RBAC: Authorizing `identity:create_grant(project_id=4076e316243946e980ce47386b18b28f, user_id=15039da209ec445d93d49e2bd6d854eb, role_id=86b73ebb032e430c9c08b78a01898cd7)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:38.178214 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148f340b-63b8-4ffd-b83b-3dface3f0f95 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:38.184231 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-0751b3eb-2618-4d24-84dd-e06432fd6e1c None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:38.185378 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0751b3eb-2618-4d24-84dd-e06432fd6e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:38.193110 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 543/1073] 10.222.0.71 () {58 vars in 1439 bytes} [Mon Jul 4 09:07:38 2022] PUT /identity/v3/projects/4076e316243946e980ce47386b18b28f/users/15039da209ec445d93d49e2bd6d854eb/roles/86b73ebb032e430c9c08b78a01898cd7 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:07:38.269183 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 531/1074] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:07:38.292894 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f340b-63b8-4ffd-b83b-3dface3f0f95 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:38.408140 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f340b-63b8-4ffd-b83b-3dface3f0f95 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:38.443504 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067c9d0-f852-4483-86e5-952a0b37a051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:38.444793 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067c9d0-f852-4483-86e5-952a0b37a051 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:38.445099 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067c9d0-f852-4483-86e5-952a0b37a051 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:38.445377 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067c9d0-f852-4483-86e5-952a0b37a051 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:38.455448 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9067c9d0-f852-4483-86e5-952a0b37a051 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:07:38.463731 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-9067c9d0-f852-4483-86e5-952a0b37a051 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:07:38.463731 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:38.463731 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:38.463731 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:38.463731 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:38.463731 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:38.463731 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:38.463731 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:38.463731 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:38.463731 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:38.463731 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:38.463731 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:38.463731 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:07:38.463731 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:07:38.463731 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:07:38.463731 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:07:38.464670 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:07:38.464670 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:07:38.464670 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:07:38.464670 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:07:38.464670 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:38.464670 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:38.464670 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:38.464670 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:38.464670 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:38.464670 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:38.464670 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:38.464670 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:38.464670 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:38.464670 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:38.464670 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:38.464670 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:38.465617 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:38.465617 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:38.465617 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:38.465617 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:38.465617 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:07:38.465617 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:07:38.465617 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:07:38.465617 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:07:38.465617 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:07:38.465617 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:07:38.465617 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:07:38.465617 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:07:38.465617 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 544/1075] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 4 09:07:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:07:38.472434 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0751b3eb-2618-4d24-84dd-e06432fd6e1c None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:38.525539 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:38.577964 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0751b3eb-2618-4d24-84dd-e06432fd6e1c None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:38.614502 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23765a06-5a0e-4f01-84cf-2fb0f992b512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:38.615173 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23765a06-5a0e-4f01-84cf-2fb0f992b512 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:38.615453 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23765a06-5a0e-4f01-84cf-2fb0f992b512 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:38.615728 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23765a06-5a0e-4f01-84cf-2fb0f992b512 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:38.616915 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23765a06-5a0e-4f01-84cf-2fb0f992b512 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:38.617263 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23765a06-5a0e-4f01-84cf-2fb0f992b512 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:38.618635 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23765a06-5a0e-4f01-84cf-2fb0f992b512 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:38.627183 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 532/1076] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:07:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:38.633599 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067c9d0-f852-4483-86e5-952a0b37a051 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:38.663983 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:38.740523 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067c9d0-f852-4483-86e5-952a0b37a051 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:38.774526 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48c608b-2b00-4386-8df6-d0cc86a54008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:38.775395 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c608b-2b00-4386-8df6-d0cc86a54008 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:38.775676 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c608b-2b00-4386-8df6-d0cc86a54008 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:38.775947 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c608b-2b00-4386-8df6-d0cc86a54008 None admin] PATH_INFO: `/v3/users/cdb7ee5dcac34293b8fd0af0f7ee0ebb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:38.777143 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48c608b-2b00-4386-8df6-d0cc86a54008 None admin] RBAC: Authorizing `identity:get_user(user_id=cdb7ee5dcac34293b8fd0af0f7ee0ebb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:38.778564 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48c608b-2b00-4386-8df6-d0cc86a54008 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:38.780364 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 545/1077] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 4 09:07:38 2022] GET /identity/v3/users/cdb7ee5dcac34293b8fd0af0f7ee0ebb => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:07:38.785885 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23765a06-5a0e-4f01-84cf-2fb0f992b512 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:38.829183 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abaac56-e999-4bb2-a2b0-ab4213fb940c None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:38.829531 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abaac56-e999-4bb2-a2b0-ab4213fb940c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:38.829753 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abaac56-e999-4bb2-a2b0-ab4213fb940c None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:38.831114 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 546/1078] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:07:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:07:38.835842 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd79acd0-9ce3-4f51-8dc7-6ebafd4aff04 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:38.836131 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd79acd0-9ce3-4f51-8dc7-6ebafd4aff04 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:38.836412 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd79acd0-9ce3-4f51-8dc7-6ebafd4aff04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:38.856259 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-bd79acd0-9ce3-4f51-8dc7-6ebafd4aff04 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:38.857775 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bd79acd0-9ce3-4f51-8dc7-6ebafd4aff04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:38.891739 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23765a06-5a0e-4f01-84cf-2fb0f992b512 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:38.902872 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:38.927846 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50a5d1b-b0bd-4ab9-81ee-fddcc1547a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:38.929302 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a5d1b-b0bd-4ab9-81ee-fddcc1547a1a None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:38.929869 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a5d1b-b0bd-4ab9-81ee-fddcc1547a1a None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:38.929869 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a5d1b-b0bd-4ab9-81ee-fddcc1547a1a None admin] PATH_INFO: `/v3/projects/97293ed76327477d86aac1c047ff78d5` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:38.931019 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50a5d1b-b0bd-4ab9-81ee-fddcc1547a1a None admin] RBAC: Authorizing `identity:get_project(project_id=97293ed76327477d86aac1c047ff78d5)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:38.932460 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50a5d1b-b0bd-4ab9-81ee-fddcc1547a1a None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:38.934344 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 533/1079] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 4 09:07:38 2022] GET /identity/v3/projects/97293ed76327477d86aac1c047ff78d5 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:38.938227 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 547/1080] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:07:38.939815 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50a5d1b-b0bd-4ab9-81ee-fddcc1547a1a None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:38.961706 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd79acd0-9ce3-4f51-8dc7-6ebafd4aff04 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:38.990330 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:39.051525 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50a5d1b-b0bd-4ab9-81ee-fddcc1547a1a None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:39.067011 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd79acd0-9ce3-4f51-8dc7-6ebafd4aff04 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:39.088668 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e2bc53-c963-49b8-a132-aa8c7bbe6ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:39.089588 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e2bc53-c963-49b8-a132-aa8c7bbe6ee6 None admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:39.089876 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e2bc53-c963-49b8-a132-aa8c7bbe6ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:39.090140 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e2bc53-c963-49b8-a132-aa8c7bbe6ee6 None admin] PATH_INFO: `/v3/projects/97293ed76327477d86aac1c047ff78d5/users/cdb7ee5dcac34293b8fd0af0f7ee0ebb/roles/f36e053b442540f08041ac7878760f65` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:39.092092 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e2bc53-c963-49b8-a132-aa8c7bbe6ee6 None admin] RBAC: Authorizing `identity:create_grant(project_id=97293ed76327477d86aac1c047ff78d5, user_id=cdb7ee5dcac34293b8fd0af0f7ee0ebb, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:39.093568 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e2bc53-c963-49b8-a132-aa8c7bbe6ee6 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:39.105129 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebd4a92-06ed-4354-bc00-9a704d2b9c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:39.105785 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd4a92-06ed-4354-bc00-9a704d2b9c93 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:39.106070 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd4a92-06ed-4354-bc00-9a704d2b9c93 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:39.106348 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd4a92-06ed-4354-bc00-9a704d2b9c93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:39.117067 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 534/1081] 10.222.0.71 () {58 vars in 1439 bytes} [Mon Jul 4 09:07:38 2022] PUT /identity/v3/projects/97293ed76327477d86aac1c047ff78d5/users/cdb7ee5dcac34293b8fd0af0f7ee0ebb/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:07:39.187969 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebd4a92-06ed-4354-bc00-9a704d2b9c93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:39.189550 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebd4a92-06ed-4354-bc00-9a704d2b9c93 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:39.220401 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:39.301161 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 548/1082] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:07:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 04 09:07:39.307388 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e2bc53-c963-49b8-a132-aa8c7bbe6ee6 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:39.352146 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:39.419625 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e2bc53-c963-49b8-a132-aa8c7bbe6ee6 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:39.454475 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a27a2f-0a0d-48b5-90b9-0a488ed4d27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:39.455268 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a27a2f-0a0d-48b5-90b9-0a488ed4d27e None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:39.455554 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a27a2f-0a0d-48b5-90b9-0a488ed4d27e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:39.455835 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a27a2f-0a0d-48b5-90b9-0a488ed4d27e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:39.467783 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a27a2f-0a0d-48b5-90b9-0a488ed4d27e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:39.469276 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a27a2f-0a0d-48b5-90b9-0a488ed4d27e None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:39.486548 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-26a27a2f-0a0d-48b5-90b9-0a488ed4d27e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:07:39.486548 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:39.486548 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:39.486548 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:39.486548 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:39.486548 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:39.486548 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:39.486548 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:39.486548 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:39.486548 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:39.486548 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:39.486548 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:39.486548 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:07:39.486548 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:07:39.486548 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:07:39.487485 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:07:39.487485 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:39.487485 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:39.487485 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:39.487485 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:39.487485 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:39.487485 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:39.487485 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:39.487485 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:39.487485 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:39.487485 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:39.487485 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:39.487485 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:39.487485 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:39.487485 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:39.487485 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:39.487485 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:39.488434 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:07:39.488434 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:07:39.488434 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:07:39.488434 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:39.492668 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 535/1083] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:07:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:07:39.494026 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebd4a92-06ed-4354-bc00-9a704d2b9c93 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:39.597761 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebd4a92-06ed-4354-bc00-9a704d2b9c93 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:39.623947 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:39.634375 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23946459-7441-4f58-9752-b5ca001fa194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:39.635043 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23946459-7441-4f58-9752-b5ca001fa194 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:39.635329 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23946459-7441-4f58-9752-b5ca001fa194 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:39.635607 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23946459-7441-4f58-9752-b5ca001fa194 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:39.636334 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23946459-7441-4f58-9752-b5ca001fa194 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:39.636645 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23946459-7441-4f58-9752-b5ca001fa194 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:39.638035 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23946459-7441-4f58-9752-b5ca001fa194 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:39.657074 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 549/1084] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:07:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:40.262044 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618bb4b-a072-4896-ae68-005cbe01ed2a None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:40.262581 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618bb4b-a072-4896-ae68-005cbe01ed2a None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:40.262581 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618bb4b-a072-4896-ae68-005cbe01ed2a None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:40.263955 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 536/1085] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:07:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:40.268905 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46567652-3311-4c36-8934-04f011162f4c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:40.269315 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46567652-3311-4c36-8934-04f011162f4c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:40.269505 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46567652-3311-4c36-8934-04f011162f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:40.286709 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-46567652-3311-4c36-8934-04f011162f4c None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:40.288772 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-46567652-3311-4c36-8934-04f011162f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:40.354935 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 550/1086] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:07:40.378112 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3618bb4b-a072-4896-ae68-005cbe01ed2a None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:40.471486 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:40.482897 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3618bb4b-a072-4896-ae68-005cbe01ed2a None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:40.515910 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb706d1-f0cb-461d-8d71-56453772b159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:40.516642 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb706d1-f0cb-461d-8d71-56453772b159 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:40.516922 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb706d1-f0cb-461d-8d71-56453772b159 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:40.517228 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb706d1-f0cb-461d-8d71-56453772b159 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:40.528496 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb706d1-f0cb-461d-8d71-56453772b159 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:07:40.536419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-edb706d1-f0cb-461d-8d71-56453772b159 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 09:07:40.536419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:40.536419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:40.536419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:40.536419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:40.536419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:40.536419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:40.536419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:40.536419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:40.536419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:40.536419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:40.536419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:40.536419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:07:40.536419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:07:40.536419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:07:40.536419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:07:40.537549 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:07:40.537549 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:07:40.537549 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:07:40.537549 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:07:40.537549 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:40.537549 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:40.537549 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:40.537549 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:40.537549 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:40.537549 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:40.537549 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:40.537549 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:40.537549 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:40.537549 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:40.537549 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:40.537549 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:40.539324 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:40.539324 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:40.539324 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:40.539324 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:40.539324 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:07:40.539324 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:07:40.539324 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:07:40.539324 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:07:40.539324 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:07:40.539324 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:07:40.539324 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 09:07:40.539324 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:40.539324 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 537/1087] 10.222.0.71 () {56 vars in 1120 bytes} [Mon Jul 4 09:07:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:07:40.544225 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46567652-3311-4c36-8934-04f011162f4c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:40.641367 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46567652-3311-4c36-8934-04f011162f4c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:40.658318 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:40.677489 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcafce7-5202-4427-9327-d9bb5b925f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:40.678161 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcafce7-5202-4427-9327-d9bb5b925f42 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:40.678611 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcafce7-5202-4427-9327-d9bb5b925f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:40.678902 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcafce7-5202-4427-9327-d9bb5b925f42 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:40.679648 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcafce7-5202-4427-9327-d9bb5b925f42 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:40.679957 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcafce7-5202-4427-9327-d9bb5b925f42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:40.681351 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcafce7-5202-4427-9327-d9bb5b925f42 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:40.689221 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 551/1088] 10.222.0.71 () {56 vars in 1115 bytes} [Mon Jul 4 09:07:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:07:40.695651 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb706d1-f0cb-461d-8d71-56453772b159 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:40.793545 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb706d1-f0cb-461d-8d71-56453772b159 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:40.833682 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763c8204-b44d-481b-8be2-ecc1a49bb150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:40.834651 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c8204-b44d-481b-8be2-ecc1a49bb150 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:40.835013 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c8204-b44d-481b-8be2-ecc1a49bb150 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:40.835540 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c8204-b44d-481b-8be2-ecc1a49bb150 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:40.848340 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763c8204-b44d-481b-8be2-ecc1a49bb150 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:40.849834 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763c8204-b44d-481b-8be2-ecc1a49bb150 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:40.867069 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-763c8204-b44d-481b-8be2-ecc1a49bb150 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:07:40.867069 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:40.867069 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:40.867069 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:40.867069 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:40.867069 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:40.867069 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:40.867069 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:40.867069 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:40.867069 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:40.867069 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:40.867069 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:40.867069 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:07:40.867069 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:07:40.867069 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:07:40.867963 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:07:40.867963 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:40.867963 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:40.867963 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:40.867963 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:40.867963 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:40.867963 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:40.867963 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:40.867963 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:40.867963 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:40.867963 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:40.867963 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:40.867963 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:40.867963 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:40.867963 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:40.867963 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:40.867963 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:40.868911 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:07:40.868911 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:07:40.868911 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:07:40.868911 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:40.868911 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 538/1089] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:07:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:40.873950 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcafce7-5202-4427-9327-d9bb5b925f42 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:40.973420 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcafce7-5202-4427-9327-d9bb5b925f42 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:41.008097 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222fbb17-d1e1-4566-a9e5-51ce9a041481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:41.008767 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222fbb17-d1e1-4566-a9e5-51ce9a041481 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:41.009075 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222fbb17-d1e1-4566-a9e5-51ce9a041481 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:41.009363 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222fbb17-d1e1-4566-a9e5-51ce9a041481 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:41.010091 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222fbb17-d1e1-4566-a9e5-51ce9a041481 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:41.010400 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222fbb17-d1e1-4566-a9e5-51ce9a041481 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:41.011763 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222fbb17-d1e1-4566-a9e5-51ce9a041481 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:41.024344 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 552/1090] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:07:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:41.030060 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763c8204-b44d-481b-8be2-ecc1a49bb150 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:41.032942 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:41.130197 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763c8204-b44d-481b-8be2-ecc1a49bb150 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:41.150335 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225d71a1-8141-4168-b30b-daa379a21fa3 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:41.150710 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225d71a1-8141-4168-b30b-daa379a21fa3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:41.151081 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225d71a1-8141-4168-b30b-daa379a21fa3 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:41.152508 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 553/1091] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:07:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:07:41.157225 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebac3ac-a9b2-4c70-b028-bf1da5207c95 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:41.157579 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebac3ac-a9b2-4c70-b028-bf1da5207c95 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:41.157924 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebac3ac-a9b2-4c70-b028-bf1da5207c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:41.163984 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73ddd0f-c002-4294-a49f-31e8e83248cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:41.165610 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ddd0f-c002-4294-a49f-31e8e83248cc None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:41.165905 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ddd0f-c002-4294-a49f-31e8e83248cc None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:41.166185 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ddd0f-c002-4294-a49f-31e8e83248cc None admin] PATH_INFO: `/v3/users/nova` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:41.173814 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73ddd0f-c002-4294-a49f-31e8e83248cc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:41.174553 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8ebac3ac-a9b2-4c70-b028-bf1da5207c95 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:41.175260 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73ddd0f-c002-4294-a49f-31e8e83248cc None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:41.175725 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8ebac3ac-a9b2-4c70-b028-bf1da5207c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:41.189692 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-b73ddd0f-c002-4294-a49f-31e8e83248cc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 04 09:07:41.189692 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:41.189692 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:41.189692 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:41.189692 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:41.189692 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:41.189692 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:41.189692 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:41.189692 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:41.189692 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:41.189692 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:41.189692 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:41.189692 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:07:41.189692 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:07:41.189692 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:07:41.189692 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:07:41.190754 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:41.190754 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:41.190754 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:07:41.190754 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:07:41.190754 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:07:41.190754 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:07:41.190754 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:41.190754 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:41.190754 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:41.190754 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:41.190754 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:41.190754 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:41.190754 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:41.190754 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:41.190754 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:41.190754 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:41.190754 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:41.190754 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:41.191723 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:41.191723 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:41.191723 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:07:41.191723 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:07:41.191723 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:07:41.191723 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:07:41.191723 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:07:41.191723 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:07:41.191723 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 04 09:07:41.191723 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:41.191723 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 539/1092] 10.222.0.71 () {56 vars in 1093 bytes} [Mon Jul 4 09:07:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:07:41.197289 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73ddd0f-c002-4294-a49f-31e8e83248cc None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:41.208108 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:41.246335 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 554/1093] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:07:41.258574 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:41.270378 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebac3ac-a9b2-4c70-b028-bf1da5207c95 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:41.297574 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73ddd0f-c002-4294-a49f-31e8e83248cc None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:41.331959 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66509e52-671e-461d-9b1b-986ae7d8e4ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:41.332630 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66509e52-671e-461d-9b1b-986ae7d8e4ca None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:41.332917 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66509e52-671e-461d-9b1b-986ae7d8e4ca None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:41.333231 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66509e52-671e-461d-9b1b-986ae7d8e4ca None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:41.334071 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66509e52-671e-461d-9b1b-986ae7d8e4ca None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:41.334389 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66509e52-671e-461d-9b1b-986ae7d8e4ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:41.336578 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66509e52-671e-461d-9b1b-986ae7d8e4ca None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:41.354969 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 540/1094] 10.222.0.71 () {56 vars in 1133 bytes} [Mon Jul 4 09:07:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:07:41.360251 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66509e52-671e-461d-9b1b-986ae7d8e4ca None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:41.374888 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebac3ac-a9b2-4c70-b028-bf1da5207c95 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:41.415491 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e2cef-e69a-49fb-8468-522e9bcf9370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:41.416287 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e2cef-e69a-49fb-8468-522e9bcf9370 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:41.416569 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e2cef-e69a-49fb-8468-522e9bcf9370 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:41.416851 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e2cef-e69a-49fb-8468-522e9bcf9370 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:41.426701 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835e2cef-e69a-49fb-8468-522e9bcf9370 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:07:41.434402 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-835e2cef-e69a-49fb-8468-522e9bcf9370 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:07:41.434402 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:41.434402 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:41.434402 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:41.434402 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:41.434402 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:41.434402 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:41.434402 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:41.434402 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:41.434402 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:41.434402 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:41.434402 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:41.434402 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:07:41.434402 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:07:41.434402 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:07:41.434402 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:07:41.435447 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:07:41.435447 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:07:41.435447 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:07:41.435447 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:07:41.435447 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:41.435447 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:41.435447 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:41.435447 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:41.435447 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:41.435447 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:41.435447 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:41.435447 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:41.435447 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:41.435447 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:41.435447 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:41.435447 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:41.436305 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:41.436305 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:41.436305 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:41.436305 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:41.436305 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:07:41.436305 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:07:41.436305 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:07:41.436305 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:07:41.436305 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:07:41.436305 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:07:41.436305 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:07:41.436305 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:07:41.436305 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 555/1095] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 4 09:07:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:07:41.441441 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e2cef-e69a-49fb-8468-522e9bcf9370 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:41.463129 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66509e52-671e-461d-9b1b-986ae7d8e4ca None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:41.470911 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:41.483014 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:41.499876 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fa3738-9879-40bc-b453-c5e2ccef58df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:41.500602 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa3738-9879-40bc-b453-c5e2ccef58df None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:41.500956 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa3738-9879-40bc-b453-c5e2ccef58df None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:41.501332 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa3738-9879-40bc-b453-c5e2ccef58df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:41.543192 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e2cef-e69a-49fb-8468-522e9bcf9370 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:41.570159 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fa3738-9879-40bc-b453-c5e2ccef58df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:41.571603 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fa3738-9879-40bc-b453-c5e2ccef58df None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:41.576587 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e03148-c9d1-4b72-b8bb-a781c1526669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:41.577283 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e03148-c9d1-4b72-b8bb-a781c1526669 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:41.577570 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e03148-c9d1-4b72-b8bb-a781c1526669 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:41.577848 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e03148-c9d1-4b72-b8bb-a781c1526669 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:41.578588 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e03148-c9d1-4b72-b8bb-a781c1526669 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:41.578892 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e03148-c9d1-4b72-b8bb-a781c1526669 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:41.580243 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e03148-c9d1-4b72-b8bb-a781c1526669 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:41.589227 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 556/1096] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:07:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:07:41.594807 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e03148-c9d1-4b72-b8bb-a781c1526669 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:41.676668 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 541/1097] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:07:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 04 09:07:41.677688 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fa3738-9879-40bc-b453-c5e2ccef58df None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:41.683441 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:41.693403 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e03148-c9d1-4b72-b8bb-a781c1526669 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:41.698193 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:41.728063 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9460b2-f890-4af4-9482-6b98e185a968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:41.728939 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9460b2-f890-4af4-9482-6b98e185a968 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:41.729253 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9460b2-f890-4af4-9482-6b98e185a968 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:41.729534 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9460b2-f890-4af4-9482-6b98e185a968 None admin] PATH_INFO: `/v3/users/cdb7ee5dcac34293b8fd0af0f7ee0ebb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:41.730741 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9460b2-f890-4af4-9482-6b98e185a968 None admin] RBAC: Authorizing `identity:get_user(user_id=cdb7ee5dcac34293b8fd0af0f7ee0ebb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:41.732170 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9460b2-f890-4af4-9482-6b98e185a968 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:41.733816 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 557/1098] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 4 09:07:41 2022] GET /identity/v3/users/cdb7ee5dcac34293b8fd0af0f7ee0ebb => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:41.739324 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9460b2-f890-4af4-9482-6b98e185a968 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:41.777210 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fa3738-9879-40bc-b453-c5e2ccef58df None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:41.811114 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82f302b-c1f3-4b19-b7ad-eba8e0704ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:41.811908 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f302b-c1f3-4b19-b7ad-eba8e0704ef0 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:41.812201 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f302b-c1f3-4b19-b7ad-eba8e0704ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:41.812475 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f302b-c1f3-4b19-b7ad-eba8e0704ef0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:41.822975 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82f302b-c1f3-4b19-b7ad-eba8e0704ef0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:41.824425 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82f302b-c1f3-4b19-b7ad-eba8e0704ef0 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:41.838976 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9460b2-f890-4af4-9482-6b98e185a968 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:41.841872 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-a82f302b-c1f3-4b19-b7ad-eba8e0704ef0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:07:41.841872 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:41.841872 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:41.841872 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:41.841872 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:41.841872 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:41.841872 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:41.841872 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:41.841872 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:41.841872 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:41.841872 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:41.841872 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:41.841872 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:07:41.841872 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:07:41.841872 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:07:41.842885 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:07:41.842885 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:41.842885 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:41.842885 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:41.842885 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:41.842885 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:41.842885 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:41.842885 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:41.842885 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:41.842885 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:41.842885 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:41.842885 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:41.842885 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:41.842885 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:41.842885 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:41.842885 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:41.842885 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:41.843825 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:07:41.843825 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:07:41.843825 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:07:41.843825 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:41.843825 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 542/1099] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:07:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:07:41.848700 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82f302b-c1f3-4b19-b7ad-eba8e0704ef0 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:41.872281 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4480f8-9f79-4e09-b104-e6c0398b7220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:41.872932 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4480f8-9f79-4e09-b104-e6c0398b7220 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:41.873246 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4480f8-9f79-4e09-b104-e6c0398b7220 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:41.873536 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4480f8-9f79-4e09-b104-e6c0398b7220 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:41.900927 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:41.918487 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:41.942108 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4480f8-9f79-4e09-b104-e6c0398b7220 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:41.943502 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4480f8-9f79-4e09-b104-e6c0398b7220 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:41.948533 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82f302b-c1f3-4b19-b7ad-eba8e0704ef0 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:41.982093 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fed9ca-faab-4168-828f-3b2ca7d89225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:41.982759 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fed9ca-faab-4168-828f-3b2ca7d89225 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:41.983048 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fed9ca-faab-4168-828f-3b2ca7d89225 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:41.983335 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fed9ca-faab-4168-828f-3b2ca7d89225 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:41.984051 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fed9ca-faab-4168-828f-3b2ca7d89225 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:41.984361 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fed9ca-faab-4168-828f-3b2ca7d89225 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:41.985755 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fed9ca-faab-4168-828f-3b2ca7d89225 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:41.997801 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 543/1100] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:07:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:07:42.003546 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fed9ca-faab-4168-828f-3b2ca7d89225 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:42.044585 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 558/1101] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:07:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:07:42.050185 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4480f8-9f79-4e09-b104-e6c0398b7220 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:42.103364 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fed9ca-faab-4168-828f-3b2ca7d89225 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:42.107255 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:42.133552 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:42.139074 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657602d3-f3f4-4131-9aff-058e3c4b3400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:42.139867 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657602d3-f3f4-4131-9aff-058e3c4b3400 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:42.140149 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657602d3-f3f4-4131-9aff-058e3c4b3400 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:42.140432 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657602d3-f3f4-4131-9aff-058e3c4b3400 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:42.147317 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657602d3-f3f4-4131-9aff-058e3c4b3400 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:42.148731 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4480f8-9f79-4e09-b104-e6c0398b7220 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:42.148841 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657602d3-f3f4-4131-9aff-058e3c4b3400 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:42.164150 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-657602d3-f3f4-4131-9aff-058e3c4b3400 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:07:42.164150 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:07:42.164150 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:07:42.164150 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:07:42.164150 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:07:42.164150 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:07:42.164150 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:07:42.164150 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:07:42.164150 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:07:42.164150 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:07:42.164150 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:07:42.164150 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:07:42.164150 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:07:42.164150 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:07:42.164150 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:07:42.165159 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:07:42.165159 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:07:42.165159 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:07:42.165159 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:07:42.165159 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:07:42.165159 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:07:42.165159 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:07:42.165159 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:07:42.165159 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:07:42.165159 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:07:42.165159 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:07:42.165159 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:07:42.165159 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:07:42.165159 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:07:42.165159 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:07:42.165159 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:07:42.165159 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:07:42.166069 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:07:42.166069 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:07:42.166069 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:07:42.166069 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:07:42.166069 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:07:42.166069 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:07:42.166069 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:07:42.166069 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:07:42.166069 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 544/1102] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 4 09:07:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:07:42.171081 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657602d3-f3f4-4131-9aff-058e3c4b3400 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:42.183880 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91830b3-1acd-4834-b725-e7918df1890d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:42.184731 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91830b3-1acd-4834-b725-e7918df1890d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:42.185118 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91830b3-1acd-4834-b725-e7918df1890d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:42.185400 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91830b3-1acd-4834-b725-e7918df1890d None admin] PATH_INFO: `/v3/projects/97293ed76327477d86aac1c047ff78d5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:42.186575 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91830b3-1acd-4834-b725-e7918df1890d None admin] RBAC: Authorizing `identity:get_project(project_id=97293ed76327477d86aac1c047ff78d5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:42.187995 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91830b3-1acd-4834-b725-e7918df1890d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:42.189721 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 559/1103] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 4 09:07:42 2022] GET /identity/v3/projects/97293ed76327477d86aac1c047ff78d5 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:07:42.195741 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91830b3-1acd-4834-b725-e7918df1890d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:42.269335 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657602d3-f3f4-4131-9aff-058e3c4b3400 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:42.294476 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91830b3-1acd-4834-b725-e7918df1890d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:42.305416 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83b275a-39aa-4f71-bd44-b8c8d6fc69ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:42.306393 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b275a-39aa-4f71-bd44-b8c8d6fc69ac None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:42.306684 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b275a-39aa-4f71-bd44-b8c8d6fc69ac None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:42.306969 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b275a-39aa-4f71-bd44-b8c8d6fc69ac None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:42.307766 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83b275a-39aa-4f71-bd44-b8c8d6fc69ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:42.308078 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83b275a-39aa-4f71-bd44-b8c8d6fc69ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:42.309562 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83b275a-39aa-4f71-bd44-b8c8d6fc69ac None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:42.321142 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 545/1104] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 4 09:07:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:07:42.321429 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:42.327208 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83b275a-39aa-4f71-bd44-b8c8d6fc69ac None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:42.330887 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a98187b-5459-4828-9989-f590ba38b416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:42.331548 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a98187b-5459-4828-9989-f590ba38b416 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:42.331836 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a98187b-5459-4828-9989-f590ba38b416 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:42.332121 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a98187b-5459-4828-9989-f590ba38b416 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:42.332960 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a98187b-5459-4828-9989-f590ba38b416 None admin] RBAC: Adding query filter params (role.id=f36e053b442540f08041ac7878760f65, scope.project.id=97293ed76327477d86aac1c047ff78d5, user.id=cdb7ee5dcac34293b8fd0af0f7ee0ebb) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:42.333336 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a98187b-5459-4828-9989-f590ba38b416 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:42.334923 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a98187b-5459-4828-9989-f590ba38b416 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:42.351460 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 560/1105] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 4 09:07:42 2022] GET /identity/v3/role_assignments?user.id=cdb7ee5dcac34293b8fd0af0f7ee0ebb&scope.project.id=97293ed76327477d86aac1c047ff78d5&role.id=f36e053b442540f08041ac7878760f65 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:07:42.400512 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:42.451534 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83b275a-39aa-4f71-bd44-b8c8d6fc69ac None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:42.490281 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a6a785-9534-4c18-88e2-5f5d6bd080cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:42.491804 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a6a785-9534-4c18-88e2-5f5d6bd080cc None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:42.492132 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a6a785-9534-4c18-88e2-5f5d6bd080cc None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:42.492410 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a6a785-9534-4c18-88e2-5f5d6bd080cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:42.493436 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a6a785-9534-4c18-88e2-5f5d6bd080cc None admin] RBAC: Adding query filter params (role.id=86b73ebb032e430c9c08b78a01898cd7, scope.project.id=4076e316243946e980ce47386b18b28f, user.id=15039da209ec445d93d49e2bd6d854eb) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:42.493905 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a6a785-9534-4c18-88e2-5f5d6bd080cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:42.495537 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a6a785-9534-4c18-88e2-5f5d6bd080cc None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:42.502117 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 546/1106] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 4 09:07:42 2022] GET /identity/v3/role_assignments?user.id=15039da209ec445d93d49e2bd6d854eb&scope.project.id=4076e316243946e980ce47386b18b28f&role.id=86b73ebb032e430c9c08b78a01898cd7 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:07:44.429683 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e56541-c301-4af3-9a9c-20dd844ffd75 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:44.430172 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e56541-c301-4af3-9a9c-20dd844ffd75 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:44.430268 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e56541-c301-4af3-9a9c-20dd844ffd75 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:44.431603 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 561/1107] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:07:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:44.436643 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0cf551-1818-461c-bdb0-ebae1bf9f182 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:44.437094 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0cf551-1818-461c-bdb0-ebae1bf9f182 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:44.437236 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0cf551-1818-461c-bdb0-ebae1bf9f182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:44.454216 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-5f0cf551-1818-461c-bdb0-ebae1bf9f182 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:44.455414 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-5f0cf551-1818-461c-bdb0-ebae1bf9f182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:44.519967 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 547/1108] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:07:44.545143 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e56541-c301-4af3-9a9c-20dd844ffd75 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:44.644042 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e56541-c301-4af3-9a9c-20dd844ffd75 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:44.676767 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d386a64-ba46-48d2-a47d-c4e8bd53f346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:44.677572 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d386a64-ba46-48d2-a47d-c4e8bd53f346 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:44.677862 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d386a64-ba46-48d2-a47d-c4e8bd53f346 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:44.678141 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d386a64-ba46-48d2-a47d-c4e8bd53f346 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:44.679371 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d386a64-ba46-48d2-a47d-c4e8bd53f346 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:44.680795 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d386a64-ba46-48d2-a47d-c4e8bd53f346 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:44.682405 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 562/1109] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:07:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:44.687971 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0cf551-1818-461c-bdb0-ebae1bf9f182 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:44.735139 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:44.786800 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0cf551-1818-461c-bdb0-ebae1bf9f182 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:44.820445 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd1ee9f-c528-46d1-92f7-bac08900d801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:44.821137 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd1ee9f-c528-46d1-92f7-bac08900d801 None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:44.821425 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd1ee9f-c528-46d1-92f7-bac08900d801 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:44.821701 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd1ee9f-c528-46d1-92f7-bac08900d801 None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:44.822402 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd1ee9f-c528-46d1-92f7-bac08900d801 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:44.823824 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd1ee9f-c528-46d1-92f7-bac08900d801 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:44.844461 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 548/1110] 10.222.0.71 () {60 vars in 1132 bytes} [Mon Jul 4 09:07:44 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:07:46.947596 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c284e2bd-e7e9-40f8-9579-dd8bf69fbd1f None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:46.948203 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c284e2bd-e7e9-40f8-9579-dd8bf69fbd1f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:46.948203 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c284e2bd-e7e9-40f8-9579-dd8bf69fbd1f None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:46.949518 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 563/1111] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:07:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:46.954715 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995813c0-2129-444d-ab67-a78b227951f5 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:46.955123 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995813c0-2129-444d-ab67-a78b227951f5 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:46.955286 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995813c0-2129-444d-ab67-a78b227951f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:46.972354 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-995813c0-2129-444d-ab67-a78b227951f5 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:46.973581 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-995813c0-2129-444d-ab67-a78b227951f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:47.039930 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 549/1112] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:07:47.063463 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c284e2bd-e7e9-40f8-9579-dd8bf69fbd1f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:47.121629 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:47.164692 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c284e2bd-e7e9-40f8-9579-dd8bf69fbd1f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:47.197662 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb69de3-ed6d-4612-b382-beca16a120aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:47.198448 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb69de3-ed6d-4612-b382-beca16a120aa None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:47.198735 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb69de3-ed6d-4612-b382-beca16a120aa None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:47.199017 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb69de3-ed6d-4612-b382-beca16a120aa None admin] PATH_INFO: `/v3/roles/5fb58444d5604bf689014e27dccacb45` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:47.200488 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb69de3-ed6d-4612-b382-beca16a120aa None admin] RBAC: Authorizing `identity:get_role(role_id=5fb58444d5604bf689014e27dccacb45)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:47.201882 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb69de3-ed6d-4612-b382-beca16a120aa None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:47.203237 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 564/1113] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 4 09:07:47 2022] GET /identity/v3/roles/5fb58444d5604bf689014e27dccacb45 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:47.208932 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995813c0-2129-444d-ab67-a78b227951f5 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:47.239701 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:47.308371 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995813c0-2129-444d-ab67-a78b227951f5 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:47.342359 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79210f82-052d-4e6a-86c1-822618475e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:47.343241 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79210f82-052d-4e6a-86c1-822618475e3f None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:47.343521 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79210f82-052d-4e6a-86c1-822618475e3f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:47.343796 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79210f82-052d-4e6a-86c1-822618475e3f None admin] PATH_INFO: `/v3/users/fcd3d1b791174bc3bd573479f8a6ee43` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:47.357510 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79210f82-052d-4e6a-86c1-822618475e3f None admin] RBAC: Authorizing `identity:get_user(user_id=fcd3d1b791174bc3bd573479f8a6ee43)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:47.358926 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79210f82-052d-4e6a-86c1-822618475e3f None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:47.360546 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 550/1114] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 4 09:07:47 2022] GET /identity/v3/users/fcd3d1b791174bc3bd573479f8a6ee43 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:47.366355 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb69de3-ed6d-4612-b382-beca16a120aa None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:47.464697 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb69de3-ed6d-4612-b382-beca16a120aa None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:47.489248 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:47.497830 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4250cf-ee00-4a46-baa0-818e5718c6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:47.498480 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4250cf-ee00-4a46-baa0-818e5718c6be None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:47.498770 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4250cf-ee00-4a46-baa0-818e5718c6be None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:47.499054 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4250cf-ee00-4a46-baa0-818e5718c6be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:47.564553 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4250cf-ee00-4a46-baa0-818e5718c6be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:47.566017 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4250cf-ee00-4a46-baa0-818e5718c6be None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:47.661692 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 565/1115] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:07:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:07:47.667668 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79210f82-052d-4e6a-86c1-822618475e3f None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:47.754812 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:47.765094 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79210f82-052d-4e6a-86c1-822618475e3f None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:47.798200 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d300d5-7e1d-46e0-af01-d72efcfe8a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:47.799046 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d300d5-7e1d-46e0-af01-d72efcfe8a45 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:47.799330 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d300d5-7e1d-46e0-af01-d72efcfe8a45 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:47.799607 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d300d5-7e1d-46e0-af01-d72efcfe8a45 None admin] PATH_INFO: `/v3/projects/97293ed76327477d86aac1c047ff78d5` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:47.800769 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d300d5-7e1d-46e0-af01-d72efcfe8a45 None admin] RBAC: Authorizing `identity:get_project(project_id=97293ed76327477d86aac1c047ff78d5)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:47.802227 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d300d5-7e1d-46e0-af01-d72efcfe8a45 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:47.804543 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 551/1116] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 4 09:07:47 2022] GET /identity/v3/projects/97293ed76327477d86aac1c047ff78d5 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:47.810494 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4250cf-ee00-4a46-baa0-818e5718c6be None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:47.837780 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:47.910277 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4250cf-ee00-4a46-baa0-818e5718c6be None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:47.945154 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d7ef80-e754-4870-a9f9-10754b6d0d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:47.945799 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d7ef80-e754-4870-a9f9-10754b6d0d74 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:47.946174 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d7ef80-e754-4870-a9f9-10754b6d0d74 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:47.946453 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d7ef80-e754-4870-a9f9-10754b6d0d74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:47.947280 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d7ef80-e754-4870-a9f9-10754b6d0d74 None admin] RBAC: Adding query filter params (role.id=5fb58444d5604bf689014e27dccacb45, scope.project.id=97293ed76327477d86aac1c047ff78d5, user.id=fcd3d1b791174bc3bd573479f8a6ee43) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:47.947595 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d7ef80-e754-4870-a9f9-10754b6d0d74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:47.949004 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d7ef80-e754-4870-a9f9-10754b6d0d74 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:47.954433 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 566/1117] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 4 09:07:47 2022] GET /identity/v3/role_assignments?user.id=fcd3d1b791174bc3bd573479f8a6ee43&scope.project.id=97293ed76327477d86aac1c047ff78d5&role.id=5fb58444d5604bf689014e27dccacb45 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:50.001476 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80babaaf-1cd6-4a29-a737-737fe8bde59b None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:50.002090 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80babaaf-1cd6-4a29-a737-737fe8bde59b None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:50.002090 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80babaaf-1cd6-4a29-a737-737fe8bde59b None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:50.003433 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 552/1118] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:07:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:50.008982 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb8f82d-4f93-4a80-82ed-34eeb70b600c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:50.009437 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb8f82d-4f93-4a80-82ed-34eeb70b600c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:50.009570 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb8f82d-4f93-4a80-82ed-34eeb70b600c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:50.027415 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0cb8f82d-4f93-4a80-82ed-34eeb70b600c None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:50.028619 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0cb8f82d-4f93-4a80-82ed-34eeb70b600c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:50.101788 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 567/1119] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:07:50.129548 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80babaaf-1cd6-4a29-a737-737fe8bde59b None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:50.256307 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80babaaf-1cd6-4a29-a737-737fe8bde59b None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:50.290634 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fc7027-6f0b-4e9f-93bb-1e6500e5e091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:50.291566 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fc7027-6f0b-4e9f-93bb-1e6500e5e091 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:50.291861 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fc7027-6f0b-4e9f-93bb-1e6500e5e091 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:50.292145 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fc7027-6f0b-4e9f-93bb-1e6500e5e091 None admin] PATH_INFO: `/v3/roles/5fb58444d5604bf689014e27dccacb45` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:50.293784 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fc7027-6f0b-4e9f-93bb-1e6500e5e091 None admin] RBAC: Authorizing `identity:get_role(role_id=5fb58444d5604bf689014e27dccacb45)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:50.295263 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fc7027-6f0b-4e9f-93bb-1e6500e5e091 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:50.296650 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 553/1120] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 4 09:07:50 2022] GET /identity/v3/roles/5fb58444d5604bf689014e27dccacb45 => generated 223 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:50.302628 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb8f82d-4f93-4a80-82ed-34eeb70b600c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:50.323682 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:50.424565 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb8f82d-4f93-4a80-82ed-34eeb70b600c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:50.469474 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d18233-bd03-4d77-ba2e-6cdff4e108da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:50.470787 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d18233-bd03-4d77-ba2e-6cdff4e108da None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:50.471071 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d18233-bd03-4d77-ba2e-6cdff4e108da None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:50.471349 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d18233-bd03-4d77-ba2e-6cdff4e108da None admin] PATH_INFO: `/v3/users/fcd3d1b791174bc3bd573479f8a6ee43` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:50.472525 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d18233-bd03-4d77-ba2e-6cdff4e108da None admin] RBAC: Authorizing `identity:get_user(user_id=fcd3d1b791174bc3bd573479f8a6ee43)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:50.473972 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d18233-bd03-4d77-ba2e-6cdff4e108da None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:50.480821 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 568/1121] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 4 09:07:50 2022] GET /identity/v3/users/fcd3d1b791174bc3bd573479f8a6ee43 => generated 286 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:07:50.486949 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fc7027-6f0b-4e9f-93bb-1e6500e5e091 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:50.514318 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:50.586641 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fc7027-6f0b-4e9f-93bb-1e6500e5e091 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:50.620272 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9bf164-4644-43c4-9983-6fef6aebd18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:50.621151 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9bf164-4644-43c4-9983-6fef6aebd18f None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:50.621440 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9bf164-4644-43c4-9983-6fef6aebd18f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:50.621719 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9bf164-4644-43c4-9983-6fef6aebd18f None admin] PATH_INFO: `/v3/projects/97293ed76327477d86aac1c047ff78d5` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:50.622899 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9bf164-4644-43c4-9983-6fef6aebd18f None admin] RBAC: Authorizing `identity:get_project(project_id=97293ed76327477d86aac1c047ff78d5)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:50.624332 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9bf164-4644-43c4-9983-6fef6aebd18f None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:50.626079 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 554/1122] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 4 09:07:50 2022] GET /identity/v3/projects/97293ed76327477d86aac1c047ff78d5 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:50.632517 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d18233-bd03-4d77-ba2e-6cdff4e108da None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:50.719754 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:50.732027 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d18233-bd03-4d77-ba2e-6cdff4e108da None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:50.767437 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82930e69-5ad5-4d85-beab-ea5ea410cce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:50.768247 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82930e69-5ad5-4d85-beab-ea5ea410cce0 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:50.768532 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82930e69-5ad5-4d85-beab-ea5ea410cce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:50.768813 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82930e69-5ad5-4d85-beab-ea5ea410cce0 None admin] PATH_INFO: `/v3/projects/97293ed76327477d86aac1c047ff78d5/users/fcd3d1b791174bc3bd573479f8a6ee43/roles/5fb58444d5604bf689014e27dccacb45` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:50.771032 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82930e69-5ad5-4d85-beab-ea5ea410cce0 None admin] RBAC: Authorizing `identity:create_grant(project_id=97293ed76327477d86aac1c047ff78d5, user_id=fcd3d1b791174bc3bd573479f8a6ee43, role_id=5fb58444d5604bf689014e27dccacb45)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:50.773128 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82930e69-5ad5-4d85-beab-ea5ea410cce0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:50.785446 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 569/1123] 10.222.0.71 () {58 vars in 1439 bytes} [Mon Jul 4 09:07:50 2022] PUT /identity/v3/projects/97293ed76327477d86aac1c047ff78d5/users/fcd3d1b791174bc3bd573479f8a6ee43/roles/5fb58444d5604bf689014e27dccacb45 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:07:52.799457 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8867b31b-576d-4fd8-946e-8378ca7d7120 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:52.800014 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8867b31b-576d-4fd8-946e-8378ca7d7120 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:52.800014 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8867b31b-576d-4fd8-946e-8378ca7d7120 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:52.801424 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 555/1124] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:07:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:52.806778 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcac65f-d527-4260-a438-af805f6b5c9b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:52.807169 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcac65f-d527-4260-a438-af805f6b5c9b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:52.807348 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcac65f-d527-4260-a438-af805f6b5c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:52.824216 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6dcac65f-d527-4260-a438-af805f6b5c9b None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:52.825452 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6dcac65f-d527-4260-a438-af805f6b5c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:52.890611 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 570/1125] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:07:52.914784 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8867b31b-576d-4fd8-946e-8378ca7d7120 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:53.007313 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:53.015093 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8867b31b-576d-4fd8-946e-8378ca7d7120 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:53.047972 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05988d85-595c-43e1-bb9f-00a9179e0c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:53.048813 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05988d85-595c-43e1-bb9f-00a9179e0c1a None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:53.049128 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05988d85-595c-43e1-bb9f-00a9179e0c1a None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:53.049413 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05988d85-595c-43e1-bb9f-00a9179e0c1a None admin] PATH_INFO: `/v3/roles/5fb58444d5604bf689014e27dccacb45` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:53.050925 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05988d85-595c-43e1-bb9f-00a9179e0c1a None admin] RBAC: Authorizing `identity:get_role(role_id=5fb58444d5604bf689014e27dccacb45)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:53.052239 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05988d85-595c-43e1-bb9f-00a9179e0c1a None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:53.054305 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 556/1126] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 4 09:07:52 2022] GET /identity/v3/roles/5fb58444d5604bf689014e27dccacb45 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:53.060039 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcac65f-d527-4260-a438-af805f6b5c9b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:53.136581 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:53.158917 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcac65f-d527-4260-a438-af805f6b5c9b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:53.191865 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec297b9-7247-471a-9f7d-7fbafbba4e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:53.192734 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec297b9-7247-471a-9f7d-7fbafbba4e70 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:53.193021 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec297b9-7247-471a-9f7d-7fbafbba4e70 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:53.193334 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec297b9-7247-471a-9f7d-7fbafbba4e70 None admin] PATH_INFO: `/v3/users/fcd3d1b791174bc3bd573479f8a6ee43` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:53.195253 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec297b9-7247-471a-9f7d-7fbafbba4e70 None admin] RBAC: Authorizing `identity:get_user(user_id=fcd3d1b791174bc3bd573479f8a6ee43)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:53.196675 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec297b9-7247-471a-9f7d-7fbafbba4e70 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:53.198336 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 571/1127] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 4 09:07:53 2022] GET /identity/v3/users/fcd3d1b791174bc3bd573479f8a6ee43 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:53.203988 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05988d85-595c-43e1-bb9f-00a9179e0c1a None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:53.300446 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05988d85-595c-43e1-bb9f-00a9179e0c1a None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:53.334003 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336cf0e0-86cb-4e7f-8f87-fe0b00f5453b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:53.334794 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336cf0e0-86cb-4e7f-8f87-fe0b00f5453b None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:53.335020 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336cf0e0-86cb-4e7f-8f87-fe0b00f5453b None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:53.335306 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336cf0e0-86cb-4e7f-8f87-fe0b00f5453b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:53.364771 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:53.402684 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336cf0e0-86cb-4e7f-8f87-fe0b00f5453b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:53.404137 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336cf0e0-86cb-4e7f-8f87-fe0b00f5453b None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:53.500587 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 557/1128] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:07:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:07:53.506549 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec297b9-7247-471a-9f7d-7fbafbba4e70 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:53.602812 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec297b9-7247-471a-9f7d-7fbafbba4e70 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:53.636373 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89251b01-5417-420b-a5bf-3f5f49ae7cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:53.637237 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89251b01-5417-420b-a5bf-3f5f49ae7cb8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:53.637519 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89251b01-5417-420b-a5bf-3f5f49ae7cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:53.637791 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89251b01-5417-420b-a5bf-3f5f49ae7cb8 None admin] PATH_INFO: `/v3/projects/97293ed76327477d86aac1c047ff78d5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:53.638963 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89251b01-5417-420b-a5bf-3f5f49ae7cb8 None admin] RBAC: Authorizing `identity:get_project(project_id=97293ed76327477d86aac1c047ff78d5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:53.640374 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89251b01-5417-420b-a5bf-3f5f49ae7cb8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:53.642056 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 572/1129] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 4 09:07:53 2022] GET /identity/v3/projects/97293ed76327477d86aac1c047ff78d5 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:53.649244 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336cf0e0-86cb-4e7f-8f87-fe0b00f5453b None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:53.709902 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:53.750079 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336cf0e0-86cb-4e7f-8f87-fe0b00f5453b None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:53.783919 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a478240-a43a-4ee1-a5c1-ca9894f51099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:53.784575 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a478240-a43a-4ee1-a5c1-ca9894f51099 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:53.784859 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a478240-a43a-4ee1-a5c1-ca9894f51099 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:53.785189 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a478240-a43a-4ee1-a5c1-ca9894f51099 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:53.786025 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a478240-a43a-4ee1-a5c1-ca9894f51099 None admin] RBAC: Adding query filter params (role.id=5fb58444d5604bf689014e27dccacb45, scope.project.id=97293ed76327477d86aac1c047ff78d5, user.id=fcd3d1b791174bc3bd573479f8a6ee43) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:07:53.786343 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a478240-a43a-4ee1-a5c1-ca9894f51099 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:53.788485 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a478240-a43a-4ee1-a5c1-ca9894f51099 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:53.794380 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 558/1130] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 4 09:07:53 2022] GET /identity/v3/role_assignments?user.id=fcd3d1b791174bc3bd573479f8a6ee43&scope.project.id=97293ed76327477d86aac1c047ff78d5&role.id=5fb58444d5604bf689014e27dccacb45 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:55.830929 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4532548-0005-42a5-866a-2acd87bd80c4 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:55.831525 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4532548-0005-42a5-866a-2acd87bd80c4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:55.831525 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4532548-0005-42a5-866a-2acd87bd80c4 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:55.832836 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 573/1131] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:07:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:55.838839 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14e2b6-2b21-4a9b-a8d9-d22ebd2f7978 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:55.839240 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14e2b6-2b21-4a9b-a8d9-d22ebd2f7978 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:55.839409 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14e2b6-2b21-4a9b-a8d9-d22ebd2f7978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:55.855918 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-4e14e2b6-2b21-4a9b-a8d9-d22ebd2f7978 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:55.857157 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4e14e2b6-2b21-4a9b-a8d9-d22ebd2f7978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:55.922043 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 559/1132] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:07:55.946172 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4532548-0005-42a5-866a-2acd87bd80c4 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:55.965733 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:56.071842 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4532548-0005-42a5-866a-2acd87bd80c4 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:56.105901 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8481c988-4112-456a-8638-c58fea1f985a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:56.106624 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8481c988-4112-456a-8638-c58fea1f985a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:56.106909 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8481c988-4112-456a-8638-c58fea1f985a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:56.107186 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8481c988-4112-456a-8638-c58fea1f985a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:56.108359 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8481c988-4112-456a-8638-c58fea1f985a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:56.109818 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8481c988-4112-456a-8638-c58fea1f985a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:56.111426 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 574/1133] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:07:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:56.117942 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14e2b6-2b21-4a9b-a8d9-d22ebd2f7978 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:56.172007 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:56.217240 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14e2b6-2b21-4a9b-a8d9-d22ebd2f7978 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:56.250679 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f6ae16-06a5-4f2d-a807-40f31d3022ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:56.251323 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f6ae16-06a5-4f2d-a807-40f31d3022ed None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:56.251605 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f6ae16-06a5-4f2d-a807-40f31d3022ed None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:56.251879 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f6ae16-06a5-4f2d-a807-40f31d3022ed None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:56.252581 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f6ae16-06a5-4f2d-a807-40f31d3022ed None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:56.254030 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f6ae16-06a5-4f2d-a807-40f31d3022ed None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:56.279990 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 560/1134] 10.222.0.71 () {60 vars in 1141 bytes} [Mon Jul 4 09:07:56 2022] POST /identity/v3/projects => generated 311 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:07:58.334065 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ad0cd-3906-4e33-b54f-9254fb0e09fc None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:58.335131 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ad0cd-3906-4e33-b54f-9254fb0e09fc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:58.335260 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ad0cd-3906-4e33-b54f-9254fb0e09fc None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:58.336662 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 575/1135] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:07:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:58.342103 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dffa564-0604-407f-8615-a596e86932f6 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:58.342494 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dffa564-0604-407f-8615-a596e86932f6 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:58.342672 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dffa564-0604-407f-8615-a596e86932f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:58.359645 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-1dffa564-0604-407f-8615-a596e86932f6 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:58.360827 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-1dffa564-0604-407f-8615-a596e86932f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:58.432081 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 561/1136] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:07:58.464336 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3ad0cd-3906-4e33-b54f-9254fb0e09fc None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:58.545282 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:58.564190 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3ad0cd-3906-4e33-b54f-9254fb0e09fc None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:58.597366 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f04e3b-c2db-4f0b-9d0a-af25e961bd02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:58.598089 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f04e3b-c2db-4f0b-9d0a-af25e961bd02 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:58.598375 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f04e3b-c2db-4f0b-9d0a-af25e961bd02 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:58.598654 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f04e3b-c2db-4f0b-9d0a-af25e961bd02 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:58.600519 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f04e3b-c2db-4f0b-9d0a-af25e961bd02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:58.602114 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f04e3b-c2db-4f0b-9d0a-af25e961bd02 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:58.603715 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 576/1137] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:07:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:07:58.609454 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dffa564-0604-407f-8615-a596e86932f6 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:58.647522 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:58.708720 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dffa564-0604-407f-8615-a596e86932f6 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:58.743280 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7452b755-d9d3-465d-8a31-9221e4371c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:58.743943 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452b755-d9d3-465d-8a31-9221e4371c10 None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:58.744229 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452b755-d9d3-465d-8a31-9221e4371c10 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:58.744507 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452b755-d9d3-465d-8a31-9221e4371c10 None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:58.745240 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7452b755-d9d3-465d-8a31-9221e4371c10 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:58.746683 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7452b755-d9d3-465d-8a31-9221e4371c10 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:58.766885 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 562/1138] 10.222.0.71 () {60 vars in 1132 bytes} [Mon Jul 4 09:07:58 2022] POST /identity/v3/users => generated 286 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:08:00.835711 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d728ed1d-51c1-4e85-b7d3-fadba5f8695e None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:00.836352 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d728ed1d-51c1-4e85-b7d3-fadba5f8695e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:00.836352 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d728ed1d-51c1-4e85-b7d3-fadba5f8695e None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:00.837635 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 577/1139] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:08:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:08:00.842959 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3880f316-73b7-4896-adb4-df745c0f2ed5 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:00.843403 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3880f316-73b7-4896-adb4-df745c0f2ed5 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:00.843512 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3880f316-73b7-4896-adb4-df745c0f2ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:00.860792 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-3880f316-73b7-4896-adb4-df745c0f2ed5 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:08:00.862027 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-3880f316-73b7-4896-adb4-df745c0f2ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:08:00.927337 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 563/1140] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:08:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:08:00.951517 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d728ed1d-51c1-4e85-b7d3-fadba5f8695e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:01.051170 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d728ed1d-51c1-4e85-b7d3-fadba5f8695e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:01.101260 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdbd67e-7b9e-4356-94c7-cbebc3b2bce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:01.101976 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbd67e-7b9e-4356-94c7-cbebc3b2bce3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:01.102262 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbd67e-7b9e-4356-94c7-cbebc3b2bce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:01.102540 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbd67e-7b9e-4356-94c7-cbebc3b2bce3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:01.112621 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdbd67e-7b9e-4356-94c7-cbebc3b2bce3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:08:01.120394 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-5cdbd67e-7b9e-4356-94c7-cbebc3b2bce3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:08:01.120394 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:08:01.120394 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:08:01.120394 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:08:01.120394 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:08:01.120394 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:08:01.120394 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:08:01.120394 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:08:01.120394 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:08:01.120394 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:08:01.120394 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:08:01.120394 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:08:01.120394 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:08:01.120394 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:08:01.120394 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:08:01.120394 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:08:01.121618 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:08:01.121618 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:08:01.121618 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:08:01.121618 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:08:01.121618 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:08:01.121618 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:08:01.121618 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:08:01.121618 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:08:01.121618 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:08:01.121618 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:08:01.121618 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:08:01.121618 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:08:01.121618 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:08:01.121618 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:08:01.121618 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:08:01.121618 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:08:01.122715 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:08:01.122715 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:08:01.122715 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:08:01.122715 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:08:01.122715 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:08:01.122715 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:08:01.122715 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:08:01.122715 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:08:01.122715 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:08:01.122715 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:08:01.122715 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:08:01.122715 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:08:01.122715 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 578/1141] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 4 09:08:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:08:01.127721 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3880f316-73b7-4896-adb4-df745c0f2ed5 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:01.153812 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:08:01.228099 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3880f316-73b7-4896-adb4-df745c0f2ed5 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:01.261783 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1d0622-1607-4cec-a293-fda2ec5792b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:01.262440 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d0622-1607-4cec-a293-fda2ec5792b3 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:01.262726 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d0622-1607-4cec-a293-fda2ec5792b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:01.263008 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d0622-1607-4cec-a293-fda2ec5792b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:01.263753 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1d0622-1607-4cec-a293-fda2ec5792b3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:08:01.264066 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1d0622-1607-4cec-a293-fda2ec5792b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:08:01.265471 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1d0622-1607-4cec-a293-fda2ec5792b3 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:08:01.273890 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 564/1142] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:08:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:08:01.279812 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdbd67e-7b9e-4356-94c7-cbebc3b2bce3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:01.292562 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:08:01.378010 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdbd67e-7b9e-4356-94c7-cbebc3b2bce3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:01.410872 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a39577-cbbc-44e4-bb30-af61dcebcbb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:01.411743 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a39577-cbbc-44e4-bb30-af61dcebcbb5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:01.412027 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a39577-cbbc-44e4-bb30-af61dcebcbb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:01.412397 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a39577-cbbc-44e4-bb30-af61dcebcbb5 None admin] PATH_INFO: `/v3/users/7cf2edf9ff234e8b86f0123b812ff446` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:01.425873 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a39577-cbbc-44e4-bb30-af61dcebcbb5 None admin] RBAC: Authorizing `identity:get_user(user_id=7cf2edf9ff234e8b86f0123b812ff446)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:08:01.427338 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a39577-cbbc-44e4-bb30-af61dcebcbb5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:08:01.429670 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 579/1143] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 4 09:08:01 2022] GET /identity/v3/users/7cf2edf9ff234e8b86f0123b812ff446 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:08:01.435557 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1d0622-1607-4cec-a293-fda2ec5792b3 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:01.520389 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:08:01.535403 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1d0622-1607-4cec-a293-fda2ec5792b3 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:01.568083 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178f25f9-24e4-499a-8fd6-1eddf7e2b1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:01.569694 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f25f9-24e4-499a-8fd6-1eddf7e2b1e7 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:01.569694 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f25f9-24e4-499a-8fd6-1eddf7e2b1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:01.570005 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f25f9-24e4-499a-8fd6-1eddf7e2b1e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:01.636320 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178f25f9-24e4-499a-8fd6-1eddf7e2b1e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:08:01.637726 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178f25f9-24e4-499a-8fd6-1eddf7e2b1e7 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:08:01.718177 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:08:01.733748 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 565/1144] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:08:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:08:01.739703 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a39577-cbbc-44e4-bb30-af61dcebcbb5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:01.804580 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:08:01.836339 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a39577-cbbc-44e4-bb30-af61dcebcbb5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:01.869622 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce6ec93-058a-4c69-81a3-20f8b2e5ca0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:01.870462 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce6ec93-058a-4c69-81a3-20f8b2e5ca0d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:01.870744 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce6ec93-058a-4c69-81a3-20f8b2e5ca0d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:01.871021 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce6ec93-058a-4c69-81a3-20f8b2e5ca0d None admin] PATH_INFO: `/v3/projects/54a61e3b5f8342e3b55ce0e3572add65` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:01.872174 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce6ec93-058a-4c69-81a3-20f8b2e5ca0d None admin] RBAC: Authorizing `identity:get_project(project_id=54a61e3b5f8342e3b55ce0e3572add65)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:08:01.873629 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce6ec93-058a-4c69-81a3-20f8b2e5ca0d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:08:01.875285 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 580/1145] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 4 09:08:01 2022] GET /identity/v3/projects/54a61e3b5f8342e3b55ce0e3572add65 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:08:01.881462 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178f25f9-24e4-499a-8fd6-1eddf7e2b1e7 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:01.979023 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178f25f9-24e4-499a-8fd6-1eddf7e2b1e7 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:02.012242 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef10cd49-8daf-433b-a623-bb2f720d6801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:02.012883 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10cd49-8daf-433b-a623-bb2f720d6801 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:02.013200 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10cd49-8daf-433b-a623-bb2f720d6801 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:02.013479 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10cd49-8daf-433b-a623-bb2f720d6801 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:02.014296 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef10cd49-8daf-433b-a623-bb2f720d6801 None admin] RBAC: Adding query filter params (role.id=f36e053b442540f08041ac7878760f65, scope.project.id=54a61e3b5f8342e3b55ce0e3572add65, user.id=7cf2edf9ff234e8b86f0123b812ff446) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:08:02.014605 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef10cd49-8daf-433b-a623-bb2f720d6801 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:08:02.016019 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef10cd49-8daf-433b-a623-bb2f720d6801 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:08:02.021235 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 566/1146] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 4 09:08:01 2022] GET /identity/v3/role_assignments?user.id=7cf2edf9ff234e8b86f0123b812ff446&scope.project.id=54a61e3b5f8342e3b55ce0e3572add65&role.id=f36e053b442540f08041ac7878760f65 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:08:04.007749 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783236ab-e1ac-4222-906a-a16704d65d50 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:04.008311 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783236ab-e1ac-4222-906a-a16704d65d50 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:04.008311 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783236ab-e1ac-4222-906a-a16704d65d50 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:04.009694 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 581/1147] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:08:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:08:04.015143 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d792612b-9e14-45e3-8ca3-6af4d727455a None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:04.015526 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d792612b-9e14-45e3-8ca3-6af4d727455a None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:04.015703 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d792612b-9e14-45e3-8ca3-6af4d727455a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:04.033725 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-d792612b-9e14-45e3-8ca3-6af4d727455a None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:08:04.034908 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d792612b-9e14-45e3-8ca3-6af4d727455a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:08:04.080376 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:08:04.101856 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 567/1148] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:08:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:08:04.124352 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783236ab-e1ac-4222-906a-a16704d65d50 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:04.227037 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783236ab-e1ac-4222-906a-a16704d65d50 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:04.259993 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac20210-136b-4203-8bfc-ab43aaeee4e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:04.260709 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac20210-136b-4203-8bfc-ab43aaeee4e1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:04.260994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac20210-136b-4203-8bfc-ab43aaeee4e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:04.261305 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac20210-136b-4203-8bfc-ab43aaeee4e1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:04.263857 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:08:04.272694 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac20210-136b-4203-8bfc-ab43aaeee4e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:08:04.280171 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-bac20210-136b-4203-8bfc-ab43aaeee4e1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:08:04.280171 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:08:04.280171 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:08:04.280171 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:08:04.280171 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:08:04.280171 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:08:04.280171 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:08:04.280171 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:08:04.280171 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:08:04.280171 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:08:04.280171 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:08:04.280171 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:08:04.280171 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:08:04.280171 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:08:04.280171 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:08:04.280171 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:08:04.281311 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:08:04.281311 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:08:04.281311 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:08:04.281311 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:08:04.281311 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:08:04.281311 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:08:04.281311 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:08:04.281311 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:08:04.281311 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:08:04.281311 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:08:04.281311 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:08:04.281311 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:08:04.281311 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:08:04.281311 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:08:04.281311 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:08:04.281311 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:08:04.287265 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:08:04.287265 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:08:04.287265 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:08:04.287265 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:08:04.287265 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:08:04.287265 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:08:04.287265 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:08:04.287265 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:08:04.287265 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:08:04.287265 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:08:04.287265 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:08:04.287265 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:08:04.287265 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 582/1149] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 4 09:08:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:08:04.288219 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d792612b-9e14-45e3-8ca3-6af4d727455a None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:04.384805 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d792612b-9e14-45e3-8ca3-6af4d727455a None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:04.417495 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38077a0-693e-47ae-bdce-a8928794dda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:04.418159 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38077a0-693e-47ae-bdce-a8928794dda6 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:04.418440 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38077a0-693e-47ae-bdce-a8928794dda6 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:04.418716 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38077a0-693e-47ae-bdce-a8928794dda6 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:04.419455 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38077a0-693e-47ae-bdce-a8928794dda6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:08:04.419763 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38077a0-693e-47ae-bdce-a8928794dda6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:08:04.421164 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38077a0-693e-47ae-bdce-a8928794dda6 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:08:04.429644 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 568/1150] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:08:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:08:04.435483 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac20210-136b-4203-8bfc-ab43aaeee4e1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:04.534855 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac20210-136b-4203-8bfc-ab43aaeee4e1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:04.567521 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497a0889-c30d-432f-8589-f7f605f5d124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:04.568380 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a0889-c30d-432f-8589-f7f605f5d124 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:04.569400 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a0889-c30d-432f-8589-f7f605f5d124 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:04.569692 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a0889-c30d-432f-8589-f7f605f5d124 None admin] PATH_INFO: `/v3/users/7cf2edf9ff234e8b86f0123b812ff446` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:04.570873 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497a0889-c30d-432f-8589-f7f605f5d124 None admin] RBAC: Authorizing `identity:get_user(user_id=7cf2edf9ff234e8b86f0123b812ff446)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:08:04.572295 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497a0889-c30d-432f-8589-f7f605f5d124 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:08:04.573921 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 583/1151] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 4 09:08:04 2022] GET /identity/v3/users/7cf2edf9ff234e8b86f0123b812ff446 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:08:04.579703 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38077a0-693e-47ae-bdce-a8928794dda6 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:04.622846 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:08:04.679449 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38077a0-693e-47ae-bdce-a8928794dda6 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:04.711445 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7f50ea-392f-4111-a7e4-38bf678339be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:04.712297 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f50ea-392f-4111-a7e4-38bf678339be None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:04.712580 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f50ea-392f-4111-a7e4-38bf678339be None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:04.713142 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f50ea-392f-4111-a7e4-38bf678339be None admin] PATH_INFO: `/v3/projects/54a61e3b5f8342e3b55ce0e3572add65` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:04.714321 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7f50ea-392f-4111-a7e4-38bf678339be None admin] RBAC: Authorizing `identity:get_project(project_id=54a61e3b5f8342e3b55ce0e3572add65)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:08:04.715757 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7f50ea-392f-4111-a7e4-38bf678339be None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:08:04.717485 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 569/1152] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 4 09:08:04 2022] GET /identity/v3/projects/54a61e3b5f8342e3b55ce0e3572add65 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:08:04.723252 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497a0889-c30d-432f-8589-f7f605f5d124 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:04.784426 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:08:04.822610 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497a0889-c30d-432f-8589-f7f605f5d124 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:04.857366 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d5391a-2d44-46d9-8c02-21f929a1160c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:04.858255 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d5391a-2d44-46d9-8c02-21f929a1160c None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:04.858540 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d5391a-2d44-46d9-8c02-21f929a1160c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:04.858819 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d5391a-2d44-46d9-8c02-21f929a1160c None admin] PATH_INFO: `/v3/projects/54a61e3b5f8342e3b55ce0e3572add65/users/7cf2edf9ff234e8b86f0123b812ff446/roles/f36e053b442540f08041ac7878760f65` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:04.860753 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d5391a-2d44-46d9-8c02-21f929a1160c None admin] RBAC: Authorizing `identity:create_grant(project_id=54a61e3b5f8342e3b55ce0e3572add65, user_id=7cf2edf9ff234e8b86f0123b812ff446, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:08:04.862230 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d5391a-2d44-46d9-8c02-21f929a1160c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:08:04.875292 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 584/1153] 10.222.0.71 () {58 vars in 1439 bytes} [Mon Jul 4 09:08:04 2022] PUT /identity/v3/projects/54a61e3b5f8342e3b55ce0e3572add65/users/7cf2edf9ff234e8b86f0123b812ff446/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:08:06.859239 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa81e4-7538-4989-bd41-84e0e846b31c None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:06.859808 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa81e4-7538-4989-bd41-84e0e846b31c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:06.859808 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa81e4-7538-4989-bd41-84e0e846b31c None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:06.861321 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 570/1154] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:08:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:08:06.866948 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39918cf1-8d28-4b7e-99b4-c61554f1eae7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:06.867391 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39918cf1-8d28-4b7e-99b4-c61554f1eae7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:06.867391 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39918cf1-8d28-4b7e-99b4-c61554f1eae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:06.884023 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-39918cf1-8d28-4b7e-99b4-c61554f1eae7 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:08:06.885229 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-39918cf1-8d28-4b7e-99b4-c61554f1eae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:08:06.951127 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 585/1155] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:08:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:08:06.975519 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfa81e4-7538-4989-bd41-84e0e846b31c None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:07.074747 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfa81e4-7538-4989-bd41-84e0e846b31c None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:07.086745 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:08:07.123779 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8b3c2f-4414-43f4-9cfd-a55c7023546c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:07.124587 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b3c2f-4414-43f4-9cfd-a55c7023546c None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:07.124873 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b3c2f-4414-43f4-9cfd-a55c7023546c None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:07.125183 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b3c2f-4414-43f4-9cfd-a55c7023546c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:07.137916 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8b3c2f-4414-43f4-9cfd-a55c7023546c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:08:07.145485 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-af8b3c2f-4414-43f4-9cfd-a55c7023546c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:08:07.145485 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:08:07.145485 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:08:07.145485 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:08:07.145485 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:08:07.145485 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:08:07.145485 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:08:07.145485 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:08:07.145485 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:08:07.145485 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:08:07.145485 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:08:07.145485 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:08:07.145485 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:08:07.145485 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:08:07.145485 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:08:07.145485 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:08:07.147293 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:08:07.147293 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:08:07.147293 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:08:07.147293 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:08:07.147293 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:08:07.147293 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:08:07.147293 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:08:07.147293 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:08:07.147293 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:08:07.147293 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:08:07.147293 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:08:07.147293 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:08:07.147293 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:08:07.147293 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:08:07.147293 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:08:07.147293 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:08:07.148363 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:08:07.148363 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:08:07.148363 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:08:07.148363 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:08:07.148363 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:08:07.148363 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:08:07.148363 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:08:07.148363 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:08:07.148363 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:08:07.148363 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:08:07.148363 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:08:07.148363 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:08:07.148363 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 571/1156] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 4 09:08:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:08:07.153498 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39918cf1-8d28-4b7e-99b4-c61554f1eae7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:07.207437 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:08:07.252330 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39918cf1-8d28-4b7e-99b4-c61554f1eae7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:07.286160 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12128b33-c5d5-49dc-9cef-001ff72508c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:07.286815 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12128b33-c5d5-49dc-9cef-001ff72508c0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:07.287095 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12128b33-c5d5-49dc-9cef-001ff72508c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:07.287832 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12128b33-c5d5-49dc-9cef-001ff72508c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:07.288569 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12128b33-c5d5-49dc-9cef-001ff72508c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:08:07.288874 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12128b33-c5d5-49dc-9cef-001ff72508c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:08:07.290252 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12128b33-c5d5-49dc-9cef-001ff72508c0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:08:07.297976 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 586/1157] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:08:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:08:07.304469 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8b3c2f-4414-43f4-9cfd-a55c7023546c None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:07.402832 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8b3c2f-4414-43f4-9cfd-a55c7023546c None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:07.436196 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be45f7f-5998-41f2-9906-a5aaa6351ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:07.437070 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be45f7f-5998-41f2-9906-a5aaa6351ffc None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:07.437359 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be45f7f-5998-41f2-9906-a5aaa6351ffc None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:07.437636 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be45f7f-5998-41f2-9906-a5aaa6351ffc None admin] PATH_INFO: `/v3/users/7cf2edf9ff234e8b86f0123b812ff446` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:07.438799 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be45f7f-5998-41f2-9906-a5aaa6351ffc None admin] RBAC: Authorizing `identity:get_user(user_id=7cf2edf9ff234e8b86f0123b812ff446)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:08:07.440224 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be45f7f-5998-41f2-9906-a5aaa6351ffc None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:08:07.441855 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 572/1158] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 4 09:08:07 2022] GET /identity/v3/users/7cf2edf9ff234e8b86f0123b812ff446 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:08:07.447789 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12128b33-c5d5-49dc-9cef-001ff72508c0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:07.545624 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12128b33-c5d5-49dc-9cef-001ff72508c0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:07.561113 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:08:07.578324 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eb33f5-b2b5-47cd-820e-27d7a9e8f035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:07.578964 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eb33f5-b2b5-47cd-820e-27d7a9e8f035 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:07.579248 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eb33f5-b2b5-47cd-820e-27d7a9e8f035 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:07.579557 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eb33f5-b2b5-47cd-820e-27d7a9e8f035 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:07.645258 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eb33f5-b2b5-47cd-820e-27d7a9e8f035 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:08:07.647245 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eb33f5-b2b5-47cd-820e-27d7a9e8f035 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:08:07.741356 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 587/1159] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:08:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:08:07.747141 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be45f7f-5998-41f2-9906-a5aaa6351ffc None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:07.786653 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:08:07.846145 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be45f7f-5998-41f2-9906-a5aaa6351ffc None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:07.879527 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec4eee7-291e-4823-ae3b-ba4f4bf76c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:07.880368 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4eee7-291e-4823-ae3b-ba4f4bf76c2a None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:07.880656 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4eee7-291e-4823-ae3b-ba4f4bf76c2a None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:07.881536 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4eee7-291e-4823-ae3b-ba4f4bf76c2a None admin] PATH_INFO: `/v3/projects/54a61e3b5f8342e3b55ce0e3572add65` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:07.882869 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec4eee7-291e-4823-ae3b-ba4f4bf76c2a None admin] RBAC: Authorizing `identity:get_project(project_id=54a61e3b5f8342e3b55ce0e3572add65)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:08:07.884292 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec4eee7-291e-4823-ae3b-ba4f4bf76c2a None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:08:07.886003 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 573/1160] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 4 09:08:07 2022] GET /identity/v3/projects/54a61e3b5f8342e3b55ce0e3572add65 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:08:07.892066 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eb33f5-b2b5-47cd-820e-27d7a9e8f035 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:07.909667 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:08:07.988488 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eb33f5-b2b5-47cd-820e-27d7a9e8f035 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:08.020962 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac07f4d1-3a0b-448d-96d6-3211982894aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:08.022193 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac07f4d1-3a0b-448d-96d6-3211982894aa None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:08.022482 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac07f4d1-3a0b-448d-96d6-3211982894aa None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:08.022761 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac07f4d1-3a0b-448d-96d6-3211982894aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:08.023589 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac07f4d1-3a0b-448d-96d6-3211982894aa None admin] RBAC: Adding query filter params (role.id=f36e053b442540f08041ac7878760f65, scope.project.id=54a61e3b5f8342e3b55ce0e3572add65, user.id=7cf2edf9ff234e8b86f0123b812ff446) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:08:08.023900 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac07f4d1-3a0b-448d-96d6-3211982894aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:08:08.025344 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac07f4d1-3a0b-448d-96d6-3211982894aa None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:08:08.031013 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 588/1161] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 4 09:08:07 2022] GET /identity/v3/role_assignments?user.id=7cf2edf9ff234e8b86f0123b812ff446&scope.project.id=54a61e3b5f8342e3b55ce0e3572add65&role.id=f36e053b442540f08041ac7878760f65 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:08:10.122200 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7df78-df06-4e95-9ce8-1c9dfdf9c49f None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:10.122738 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7df78-df06-4e95-9ce8-1c9dfdf9c49f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:10.122738 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7df78-df06-4e95-9ce8-1c9dfdf9c49f None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:10.124120 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 574/1162] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:08:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:08:10.129354 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69155509-2578-4754-ace7-fb1ac0913e44 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:10.129757 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69155509-2578-4754-ace7-fb1ac0913e44 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:10.129921 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69155509-2578-4754-ace7-fb1ac0913e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:10.147355 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-69155509-2578-4754-ace7-fb1ac0913e44 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:08:10.148624 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-69155509-2578-4754-ace7-fb1ac0913e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:08:10.214260 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 589/1163] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:08:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:08:10.237240 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f7df78-df06-4e95-9ce8-1c9dfdf9c49f None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:10.336649 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f7df78-df06-4e95-9ce8-1c9dfdf9c49f None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:10.343940 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:08:10.370219 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff06ff98-ad14-4f6a-a633-7639903e8394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:10.370854 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff06ff98-ad14-4f6a-a633-7639903e8394 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:10.371138 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff06ff98-ad14-4f6a-a633-7639903e8394 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:10.371416 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff06ff98-ad14-4f6a-a633-7639903e8394 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:10.436543 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff06ff98-ad14-4f6a-a633-7639903e8394 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:08:10.438032 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff06ff98-ad14-4f6a-a633-7639903e8394 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:08:10.534903 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 575/1164] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:08:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:08:10.541432 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69155509-2578-4754-ace7-fb1ac0913e44 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:10.552660 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:08:10.638683 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69155509-2578-4754-ace7-fb1ac0913e44 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:10.671879 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9082464b-cd6d-41c2-812e-7959fe576f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:10.672612 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9082464b-cd6d-41c2-812e-7959fe576f81 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:10.672900 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9082464b-cd6d-41c2-812e-7959fe576f81 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:10.673218 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9082464b-cd6d-41c2-812e-7959fe576f81 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:10.679895 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9082464b-cd6d-41c2-812e-7959fe576f81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:08:10.681366 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9082464b-cd6d-41c2-812e-7959fe576f81 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:08:10.695829 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-9082464b-cd6d-41c2-812e-7959fe576f81 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 04 09:08:10.695829 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:08:10.695829 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 04 09:08:10.695829 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 04 09:08:10.695829 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:08:10.695829 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:08:10.695829 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:08:10.695829 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:08:10.695829 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 04 09:08:10.695829 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:08:10.695829 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:08:10.695829 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:08:10.695829 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:08:10.695829 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:08:10.695829 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:08:10.695829 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:08:10.695829 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:08:10.697005 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:08:10.697005 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:08:10.697005 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:08:10.697005 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:08:10.697005 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:08:10.697005 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:08:10.697005 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:08:10.697005 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:08:10.697005 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:08:10.697005 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:08:10.697005 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:08:10.697005 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:08:10.697005 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:08:10.697005 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:08:10.697005 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:08:10.697005 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:08:10.697923 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:08:10.697923 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:08:10.697923 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:08:10.697923 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:08:10.697923 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:08:10.697923 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:08:10.697923 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:08:10.697923 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:08:10.697923 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:08:10.697923 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:08:10.697923 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 04 09:08:10.697923 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:08:10.697923 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 04 09:08:10.697923 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:08:10.697923 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 590/1165] 10.222.0.71 () {56 vars in 1117 bytes} [Mon Jul 4 09:08:10 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:08:10.703101 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff06ff98-ad14-4f6a-a633-7639903e8394 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:10.711747 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:08:10.799974 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff06ff98-ad14-4f6a-a633-7639903e8394 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:10.833250 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f6255-cb00-4498-8a55-507d8df7866f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:10.833907 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f6255-cb00-4498-8a55-507d8df7866f None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:10.834195 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f6255-cb00-4498-8a55-507d8df7866f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:10.834474 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f6255-cb00-4498-8a55-507d8df7866f None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:10.835190 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998f6255-cb00-4498-8a55-507d8df7866f None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:08:10.835580 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998f6255-cb00-4498-8a55-507d8df7866f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:08:10.837632 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998f6255-cb00-4498-8a55-507d8df7866f None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:08:10.844851 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 576/1166] 10.222.0.71 () {56 vars in 1115 bytes} [Mon Jul 4 09:08:10 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:08:12.842488 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c72bb9-d89a-4dea-bf74-070e2c30c081 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:12.843029 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c72bb9-d89a-4dea-bf74-070e2c30c081 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:12.843029 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c72bb9-d89a-4dea-bf74-070e2c30c081 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:12.844399 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 591/1167] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:08:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:08:12.849521 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee3741-d777-4635-8ada-ca9e3949f2d8 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:12.849945 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee3741-d777-4635-8ada-ca9e3949f2d8 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:12.850078 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee3741-d777-4635-8ada-ca9e3949f2d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:12.866583 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-eeee3741-d777-4635-8ada-ca9e3949f2d8 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:08:12.868368 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-eeee3741-d777-4635-8ada-ca9e3949f2d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:08:12.930328 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:08:12.935577 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 577/1168] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:08:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:08:12.958719 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c72bb9-d89a-4dea-bf74-070e2c30c081 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:13.028093 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:08:13.060622 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c72bb9-d89a-4dea-bf74-070e2c30c081 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:13.092992 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1df63e-a829-4e56-a2af-b00d8f6333cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:13.093612 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1df63e-a829-4e56-a2af-b00d8f6333cb None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:13.093896 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1df63e-a829-4e56-a2af-b00d8f6333cb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:13.094176 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1df63e-a829-4e56-a2af-b00d8f6333cb None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:13.094920 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1df63e-a829-4e56-a2af-b00d8f6333cb None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:08:13.096278 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1df63e-a829-4e56-a2af-b00d8f6333cb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:08:13.110426 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 592/1169] 10.222.0.71 () {60 vars in 1138 bytes} [Mon Jul 4 09:08:12 2022] POST /identity/v3/domains => generated 267 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:08:15.146920 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871371a-bbdb-4fbb-bde9-37a9281941cc None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:15.147496 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871371a-bbdb-4fbb-bde9-37a9281941cc None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:15.147496 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871371a-bbdb-4fbb-bde9-37a9281941cc None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:15.148970 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 578/1170] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:08:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:08:15.154079 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa303be-de67-42ea-9ac7-4ea26066270f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:15.154465 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa303be-de67-42ea-9ac7-4ea26066270f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:15.154643 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa303be-de67-42ea-9ac7-4ea26066270f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:15.172188 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-dfa303be-de67-42ea-9ac7-4ea26066270f None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:08:15.173408 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dfa303be-de67-42ea-9ac7-4ea26066270f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:08:15.239028 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 593/1171] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:08:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:08:15.263920 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4871371a-bbdb-4fbb-bde9-37a9281941cc None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:15.363677 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4871371a-bbdb-4fbb-bde9-37a9281941cc None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:15.397201 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a87da80-ee2c-46bd-8b09-39eaeea7890f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:15.398058 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a87da80-ee2c-46bd-8b09-39eaeea7890f None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:15.398809 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a87da80-ee2c-46bd-8b09-39eaeea7890f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:15.399088 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a87da80-ee2c-46bd-8b09-39eaeea7890f None admin] PATH_INFO: `/v3/domains/7c2779688424422380b28da33c3a6c41` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:15.409265 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a87da80-ee2c-46bd-8b09-39eaeea7890f None admin] RBAC: Authorizing `identity:get_domain(domain_id=7c2779688424422380b28da33c3a6c41)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:08:15.410696 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a87da80-ee2c-46bd-8b09-39eaeea7890f None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:08:15.412326 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 579/1172] 10.222.0.71 () {56 vars in 1183 bytes} [Mon Jul 4 09:08:15 2022] GET /identity/v3/domains/7c2779688424422380b28da33c3a6c41 => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:08:15.418671 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa303be-de67-42ea-9ac7-4ea26066270f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:15.475588 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:08:15.516492 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa303be-de67-42ea-9ac7-4ea26066270f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:15.549629 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884f0d14-e23c-4dd4-9cee-c7a0aa64bd6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:15.550281 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f0d14-e23c-4dd4-9cee-c7a0aa64bd6f None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:15.550567 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f0d14-e23c-4dd4-9cee-c7a0aa64bd6f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:15.550845 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f0d14-e23c-4dd4-9cee-c7a0aa64bd6f None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:15.551548 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884f0d14-e23c-4dd4-9cee-c7a0aa64bd6f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:08:15.552965 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884f0d14-e23c-4dd4-9cee-c7a0aa64bd6f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:08:15.576629 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 594/1173] 10.222.0.71 () {60 vars in 1141 bytes} [Mon Jul 4 09:08:15 2022] POST /identity/v3/projects => generated 361 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:08:17.658081 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cdcd90-428a-463a-8fa4-a55c256ed87a None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:17.658632 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cdcd90-428a-463a-8fa4-a55c256ed87a None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:17.658632 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cdcd90-428a-463a-8fa4-a55c256ed87a None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:17.659919 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 580/1174] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:08:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:08:17.665235 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125caed4-c574-4440-bc23-c7f6c52d864a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:17.665662 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125caed4-c574-4440-bc23-c7f6c52d864a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:17.665803 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125caed4-c574-4440-bc23-c7f6c52d864a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:17.682089 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-125caed4-c574-4440-bc23-c7f6c52d864a None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:08:17.683274 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-125caed4-c574-4440-bc23-c7f6c52d864a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:08:17.748822 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 595/1175] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:08:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:08:17.774249 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cdcd90-428a-463a-8fa4-a55c256ed87a None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:17.832853 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:08:17.873577 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cdcd90-428a-463a-8fa4-a55c256ed87a None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:17.907529 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aac0a4-7ad5-4e03-b942-b7386c562f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:17.908429 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aac0a4-7ad5-4e03-b942-b7386c562f47 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:17.908660 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aac0a4-7ad5-4e03-b942-b7386c562f47 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:17.908937 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aac0a4-7ad5-4e03-b942-b7386c562f47 None admin] PATH_INFO: `/v3/domains/7c2779688424422380b28da33c3a6c41` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:17.910130 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aac0a4-7ad5-4e03-b942-b7386c562f47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=7c2779688424422380b28da33c3a6c41)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:08:17.911551 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aac0a4-7ad5-4e03-b942-b7386c562f47 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:08:17.913188 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 581/1176] 10.222.0.71 () {56 vars in 1183 bytes} [Mon Jul 4 09:08:17 2022] GET /identity/v3/domains/7c2779688424422380b28da33c3a6c41 => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:08:17.918973 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125caed4-c574-4440-bc23-c7f6c52d864a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:17.960665 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:08:18.015712 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125caed4-c574-4440-bc23-c7f6c52d864a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:18.047773 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ad28cc-ebeb-45d3-9f2e-f54cc681a662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:18.048428 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad28cc-ebeb-45d3-9f2e-f54cc681a662 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:18.048711 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad28cc-ebeb-45d3-9f2e-f54cc681a662 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:18.048988 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad28cc-ebeb-45d3-9f2e-f54cc681a662 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:18.049717 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ad28cc-ebeb-45d3-9f2e-f54cc681a662 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:08:18.051133 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ad28cc-ebeb-45d3-9f2e-f54cc681a662 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:08:18.070841 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 596/1177] 10.222.0.71 () {60 vars in 1132 bytes} [Mon Jul 4 09:08:17 2022] POST /identity/v3/users => generated 311 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:08:20.118956 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e018a06-9861-4cf5-810f-90b27fbc6d33 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:20.119544 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e018a06-9861-4cf5-810f-90b27fbc6d33 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:20.119544 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e018a06-9861-4cf5-810f-90b27fbc6d33 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:20.120885 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 582/1178] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:08:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:08:20.126020 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888eca60-6881-46f0-b783-f909fdd0d607 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:20.126458 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888eca60-6881-46f0-b783-f909fdd0d607 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:20.126585 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888eca60-6881-46f0-b783-f909fdd0d607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:20.142920 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-888eca60-6881-46f0-b783-f909fdd0d607 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:08:20.144103 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-888eca60-6881-46f0-b783-f909fdd0d607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:08:20.208484 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 597/1179] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:08:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:08:20.233238 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e018a06-9861-4cf5-810f-90b27fbc6d33 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:20.330899 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e018a06-9861-4cf5-810f-90b27fbc6d33 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:20.357143 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:08:20.365338 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bf5ba0-0f4a-4af2-850b-ccd7c9428e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:20.366126 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bf5ba0-0f4a-4af2-850b-ccd7c9428e4a None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:20.366415 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bf5ba0-0f4a-4af2-850b-ccd7c9428e4a None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:20.366725 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bf5ba0-0f4a-4af2-850b-ccd7c9428e4a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:20.378456 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bf5ba0-0f4a-4af2-850b-ccd7c9428e4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:08:20.386746 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-53bf5ba0-0f4a-4af2-850b-ccd7c9428e4a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:08:20.386746 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:08:20.386746 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:08:20.386746 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:08:20.386746 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:08:20.386746 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:08:20.386746 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:08:20.386746 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:08:20.386746 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:08:20.386746 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:08:20.386746 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:08:20.386746 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:08:20.386746 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:08:20.386746 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:08:20.386746 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:08:20.386746 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:08:20.387969 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:08:20.387969 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:08:20.387969 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:08:20.387969 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:08:20.387969 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:08:20.387969 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:08:20.387969 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:08:20.387969 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:08:20.387969 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:08:20.387969 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:08:20.387969 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:08:20.387969 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:08:20.387969 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:08:20.387969 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:08:20.387969 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:08:20.387969 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:08:20.389011 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:08:20.389011 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:08:20.389011 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:08:20.389011 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:08:20.389011 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:08:20.389011 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:08:20.389011 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:08:20.389011 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:08:20.389011 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:08:20.389011 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:08:20.389011 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:08:20.389011 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:08:20.389011 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 583/1180] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 4 09:08:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:08:20.393995 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888eca60-6881-46f0-b783-f909fdd0d607 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:20.421743 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:08:20.491844 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888eca60-6881-46f0-b783-f909fdd0d607 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:20.524827 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799f950f-43f3-4c08-ba45-338a4bedd810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:20.525511 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799f950f-43f3-4c08-ba45-338a4bedd810 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:20.525795 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799f950f-43f3-4c08-ba45-338a4bedd810 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:20.526071 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799f950f-43f3-4c08-ba45-338a4bedd810 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:20.526808 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799f950f-43f3-4c08-ba45-338a4bedd810 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:08:20.527112 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799f950f-43f3-4c08-ba45-338a4bedd810 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:08:20.528467 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799f950f-43f3-4c08-ba45-338a4bedd810 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:08:20.536382 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 598/1181] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:08:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:08:20.542236 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bf5ba0-0f4a-4af2-850b-ccd7c9428e4a None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:20.638033 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bf5ba0-0f4a-4af2-850b-ccd7c9428e4a None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:20.670271 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c07aa5-29ac-47e9-8cf2-3e7e53be8b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:20.671123 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c07aa5-29ac-47e9-8cf2-3e7e53be8b0d None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:20.671401 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c07aa5-29ac-47e9-8cf2-3e7e53be8b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:20.671676 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c07aa5-29ac-47e9-8cf2-3e7e53be8b0d None admin] PATH_INFO: `/v3/users/85d261cffc4746e7a4815c14cdcca9ed` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:20.684201 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c07aa5-29ac-47e9-8cf2-3e7e53be8b0d None admin] RBAC: Authorizing `identity:get_user(user_id=85d261cffc4746e7a4815c14cdcca9ed)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:08:20.685666 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c07aa5-29ac-47e9-8cf2-3e7e53be8b0d None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:08:20.687276 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 584/1182] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 4 09:08:20 2022] GET /identity/v3/users/85d261cffc4746e7a4815c14cdcca9ed => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:08:20.692978 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799f950f-43f3-4c08-ba45-338a4bedd810 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:20.783723 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:08:20.793335 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799f950f-43f3-4c08-ba45-338a4bedd810 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:20.826711 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455e2ba6-655a-4d5a-b352-11344b1151e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:20.827354 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e2ba6-655a-4d5a-b352-11344b1151e7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:20.827633 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e2ba6-655a-4d5a-b352-11344b1151e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:20.827904 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e2ba6-655a-4d5a-b352-11344b1151e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:20.893239 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455e2ba6-655a-4d5a-b352-11344b1151e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:08:20.894600 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455e2ba6-655a-4d5a-b352-11344b1151e7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:08:20.980488 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:08:20.990850 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 599/1183] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:08:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:08:20.997631 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c07aa5-29ac-47e9-8cf2-3e7e53be8b0d None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:21.042786 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:08:21.094303 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c07aa5-29ac-47e9-8cf2-3e7e53be8b0d None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:21.126490 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4e2f8b-0b7c-4522-a149-000ce36b90d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:21.127331 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e2f8b-0b7c-4522-a149-000ce36b90d5 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:21.127613 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e2f8b-0b7c-4522-a149-000ce36b90d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:21.127888 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e2f8b-0b7c-4522-a149-000ce36b90d5 None admin] PATH_INFO: `/v3/projects/4d5651ce77484ab9beca753ab22dfbac` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:21.129070 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4e2f8b-0b7c-4522-a149-000ce36b90d5 None admin] RBAC: Authorizing `identity:get_project(project_id=4d5651ce77484ab9beca753ab22dfbac)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:08:21.130494 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4e2f8b-0b7c-4522-a149-000ce36b90d5 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:08:21.132153 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 585/1184] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 4 09:08:20 2022] GET /identity/v3/projects/4d5651ce77484ab9beca753ab22dfbac => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:08:21.138634 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455e2ba6-655a-4d5a-b352-11344b1151e7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:21.236052 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455e2ba6-655a-4d5a-b352-11344b1151e7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:21.273874 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcf6a43-5fcb-4f0d-8273-2a8a3930c4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:21.274781 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf6a43-5fcb-4f0d-8273-2a8a3930c4a6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:21.275216 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf6a43-5fcb-4f0d-8273-2a8a3930c4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:21.275649 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf6a43-5fcb-4f0d-8273-2a8a3930c4a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:21.276975 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcf6a43-5fcb-4f0d-8273-2a8a3930c4a6 None admin] RBAC: Adding query filter params (role.id=f36e053b442540f08041ac7878760f65, scope.project.id=4d5651ce77484ab9beca753ab22dfbac, user.id=85d261cffc4746e7a4815c14cdcca9ed) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:08:21.277501 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcf6a43-5fcb-4f0d-8273-2a8a3930c4a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:08:21.279771 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcf6a43-5fcb-4f0d-8273-2a8a3930c4a6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:08:21.287796 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 600/1185] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 4 09:08:21 2022] GET /identity/v3/role_assignments?user.id=85d261cffc4746e7a4815c14cdcca9ed&scope.project.id=4d5651ce77484ab9beca753ab22dfbac&role.id=f36e053b442540f08041ac7878760f65 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:08:23.280637 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5999386-4cb5-4909-a805-78a4d695bb7a None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:23.281249 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5999386-4cb5-4909-a805-78a4d695bb7a None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:23.281249 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5999386-4cb5-4909-a805-78a4d695bb7a None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:23.282566 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 586/1186] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:08:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:08:23.287646 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d5012-8a20-4d9c-a9b3-7741fcbbe242 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:23.288029 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d5012-8a20-4d9c-a9b3-7741fcbbe242 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:23.288190 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d5012-8a20-4d9c-a9b3-7741fcbbe242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:23.305875 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-542d5012-8a20-4d9c-a9b3-7741fcbbe242 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:08:23.307045 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-542d5012-8a20-4d9c-a9b3-7741fcbbe242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:08:23.356815 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:08:23.374223 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 601/1187] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:08:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:08:23.397194 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5999386-4cb5-4909-a805-78a4d695bb7a None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:23.496755 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5999386-4cb5-4909-a805-78a4d695bb7a None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:23.510817 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:08:23.531415 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853281b7-519b-40ec-877a-57aa1c4f689a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:23.532136 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853281b7-519b-40ec-877a-57aa1c4f689a None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:23.532420 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853281b7-519b-40ec-877a-57aa1c4f689a None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:23.532695 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853281b7-519b-40ec-877a-57aa1c4f689a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:23.542486 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853281b7-519b-40ec-877a-57aa1c4f689a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:08:23.549981 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-853281b7-519b-40ec-877a-57aa1c4f689a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:08:23.549981 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:08:23.549981 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:08:23.549981 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:08:23.549981 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:08:23.549981 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:08:23.549981 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:08:23.549981 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:08:23.549981 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:08:23.549981 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:08:23.549981 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:08:23.549981 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:08:23.549981 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:08:23.549981 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:08:23.549981 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:08:23.549981 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:08:23.551089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:08:23.551089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:08:23.551089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:08:23.551089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:08:23.551089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:08:23.551089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:08:23.551089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:08:23.551089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:08:23.551089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:08:23.551089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:08:23.551089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:08:23.551089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:08:23.551089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:08:23.551089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:08:23.551089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:08:23.551089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:08:23.552173 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:08:23.552173 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:08:23.552173 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:08:23.552173 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:08:23.552173 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:08:23.552173 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:08:23.552173 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:08:23.552173 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:08:23.552173 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:08:23.552173 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:08:23.552173 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:08:23.552173 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:08:23.552173 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 587/1188] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 4 09:08:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:08:23.559188 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542d5012-8a20-4d9c-a9b3-7741fcbbe242 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:23.654384 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542d5012-8a20-4d9c-a9b3-7741fcbbe242 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:23.686512 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b31c92-0f9f-44b5-afcb-68ef6c1f743f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:23.687165 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b31c92-0f9f-44b5-afcb-68ef6c1f743f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:23.687446 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b31c92-0f9f-44b5-afcb-68ef6c1f743f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:23.687722 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b31c92-0f9f-44b5-afcb-68ef6c1f743f None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:23.688452 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b31c92-0f9f-44b5-afcb-68ef6c1f743f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:08:23.688758 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b31c92-0f9f-44b5-afcb-68ef6c1f743f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:08:23.690137 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b31c92-0f9f-44b5-afcb-68ef6c1f743f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:08:23.698750 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 602/1189] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:08:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:08:23.704391 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853281b7-519b-40ec-877a-57aa1c4f689a None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:23.801123 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853281b7-519b-40ec-877a-57aa1c4f689a None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:23.833871 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb90275-30c9-4115-967f-ec74a5913834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:23.834716 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb90275-30c9-4115-967f-ec74a5913834 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:23.834995 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb90275-30c9-4115-967f-ec74a5913834 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:23.835288 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb90275-30c9-4115-967f-ec74a5913834 None admin] PATH_INFO: `/v3/users/85d261cffc4746e7a4815c14cdcca9ed` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:23.836631 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb90275-30c9-4115-967f-ec74a5913834 None admin] RBAC: Authorizing `identity:get_user(user_id=85d261cffc4746e7a4815c14cdcca9ed)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:08:23.838351 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb90275-30c9-4115-967f-ec74a5913834 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:08:23.839954 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 588/1190] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 4 09:08:23 2022] GET /identity/v3/users/85d261cffc4746e7a4815c14cdcca9ed => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:08:23.845593 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b31c92-0f9f-44b5-afcb-68ef6c1f743f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:23.891228 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:08:23.943201 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b31c92-0f9f-44b5-afcb-68ef6c1f743f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:23.975950 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54bcbf0-01aa-42d0-ad27-900c953bca7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:23.976787 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54bcbf0-01aa-42d0-ad27-900c953bca7a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:23.977087 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54bcbf0-01aa-42d0-ad27-900c953bca7a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:23.977369 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54bcbf0-01aa-42d0-ad27-900c953bca7a None admin] PATH_INFO: `/v3/projects/4d5651ce77484ab9beca753ab22dfbac` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:23.978539 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54bcbf0-01aa-42d0-ad27-900c953bca7a None admin] RBAC: Authorizing `identity:get_project(project_id=4d5651ce77484ab9beca753ab22dfbac)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:08:23.979959 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54bcbf0-01aa-42d0-ad27-900c953bca7a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:08:23.981649 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 603/1191] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 4 09:08:23 2022] GET /identity/v3/projects/4d5651ce77484ab9beca753ab22dfbac => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:08:23.987595 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb90275-30c9-4115-967f-ec74a5913834 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:24.030162 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:08:24.087236 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb90275-30c9-4115-967f-ec74a5913834 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:24.119887 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810566fd-cab3-4835-b31e-8c255d03d747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:24.120767 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810566fd-cab3-4835-b31e-8c255d03d747 None admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:24.121079 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810566fd-cab3-4835-b31e-8c255d03d747 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:24.121356 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810566fd-cab3-4835-b31e-8c255d03d747 None admin] PATH_INFO: `/v3/projects/4d5651ce77484ab9beca753ab22dfbac/users/85d261cffc4746e7a4815c14cdcca9ed/roles/f36e053b442540f08041ac7878760f65` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:24.123289 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810566fd-cab3-4835-b31e-8c255d03d747 None admin] RBAC: Authorizing `identity:create_grant(project_id=4d5651ce77484ab9beca753ab22dfbac, user_id=85d261cffc4746e7a4815c14cdcca9ed, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:08:24.124719 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810566fd-cab3-4835-b31e-8c255d03d747 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:08:24.138334 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 589/1192] 10.222.0.71 () {58 vars in 1439 bytes} [Mon Jul 4 09:08:23 2022] PUT /identity/v3/projects/4d5651ce77484ab9beca753ab22dfbac/users/85d261cffc4746e7a4815c14cdcca9ed/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:08:26.125208 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afce876-4954-4606-9085-d8c30f9c7139 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:26.125775 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afce876-4954-4606-9085-d8c30f9c7139 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:26.125775 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afce876-4954-4606-9085-d8c30f9c7139 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:26.127121 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 604/1193] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:08:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:08:26.132235 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4924cc4d-3c8d-4d2e-9c1c-3b08230b6552 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:26.132692 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4924cc4d-3c8d-4d2e-9c1c-3b08230b6552 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:26.132801 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4924cc4d-3c8d-4d2e-9c1c-3b08230b6552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:26.149867 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-4924cc4d-3c8d-4d2e-9c1c-3b08230b6552 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:08:26.151057 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4924cc4d-3c8d-4d2e-9c1c-3b08230b6552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:08:26.216556 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 590/1194] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:08:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:08:26.240218 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afce876-4954-4606-9085-d8c30f9c7139 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:26.339859 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afce876-4954-4606-9085-d8c30f9c7139 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:26.353790 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:08:26.374208 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e6f23e-a4ca-4afb-a374-a84754d628cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:26.374989 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e6f23e-a4ca-4afb-a374-a84754d628cc None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:26.375323 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e6f23e-a4ca-4afb-a374-a84754d628cc None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:26.375550 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e6f23e-a4ca-4afb-a374-a84754d628cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:26.386289 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e6f23e-a4ca-4afb-a374-a84754d628cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:08:26.393926 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-59e6f23e-a4ca-4afb-a374-a84754d628cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:08:26.393926 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:08:26.393926 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:08:26.393926 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:08:26.393926 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:08:26.393926 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:08:26.393926 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:08:26.393926 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:08:26.393926 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:08:26.393926 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:08:26.393926 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:08:26.393926 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:08:26.393926 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:08:26.393926 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:08:26.393926 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:08:26.393926 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:08:26.395013 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:08:26.395013 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:08:26.395013 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:08:26.395013 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:08:26.395013 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:08:26.395013 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:08:26.395013 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:08:26.395013 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:08:26.395013 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:08:26.395013 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:08:26.395013 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:08:26.395013 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:08:26.395013 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:08:26.395013 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:08:26.395013 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:08:26.395013 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:08:26.396055 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:08:26.396055 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:08:26.396055 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:08:26.396055 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:08:26.396055 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:08:26.396055 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:08:26.396055 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:08:26.396055 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:08:26.396055 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:08:26.396055 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:08:26.396055 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:08:26.396055 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:08:26.396055 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 605/1195] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 4 09:08:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:08:26.401953 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4924cc4d-3c8d-4d2e-9c1c-3b08230b6552 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:26.432491 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:08:26.500184 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4924cc4d-3c8d-4d2e-9c1c-3b08230b6552 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:26.533507 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ca0edc-47f8-4479-bce2-97257c8c94df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:26.534174 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca0edc-47f8-4479-bce2-97257c8c94df None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:26.534466 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca0edc-47f8-4479-bce2-97257c8c94df None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:26.534747 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca0edc-47f8-4479-bce2-97257c8c94df None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:26.535491 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ca0edc-47f8-4479-bce2-97257c8c94df None admin] RBAC: Adding query filter params (name=admin) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:08:26.535803 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ca0edc-47f8-4479-bce2-97257c8c94df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:08:26.537195 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ca0edc-47f8-4479-bce2-97257c8c94df None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:08:26.545104 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 591/1196] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:08:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:08:26.550879 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e6f23e-a4ca-4afb-a374-a84754d628cc None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:26.648509 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e6f23e-a4ca-4afb-a374-a84754d628cc None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:26.681084 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4b3fea-dc5f-4f57-905b-dd9e60c6e6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:26.681938 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b3fea-dc5f-4f57-905b-dd9e60c6e6af None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:26.682218 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b3fea-dc5f-4f57-905b-dd9e60c6e6af None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:26.682492 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b3fea-dc5f-4f57-905b-dd9e60c6e6af None admin] PATH_INFO: `/v3/users/85d261cffc4746e7a4815c14cdcca9ed` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:26.683642 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4b3fea-dc5f-4f57-905b-dd9e60c6e6af None admin] RBAC: Authorizing `identity:get_user(user_id=85d261cffc4746e7a4815c14cdcca9ed)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:08:26.685114 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4b3fea-dc5f-4f57-905b-dd9e60c6e6af None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:08:26.686699 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 606/1197] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 4 09:08:26 2022] GET /identity/v3/users/85d261cffc4746e7a4815c14cdcca9ed => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:08:26.692388 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ca0edc-47f8-4479-bce2-97257c8c94df None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:26.783775 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:08:26.791483 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ca0edc-47f8-4479-bce2-97257c8c94df None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:26.824576 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc79bc7-3e8a-4bba-9519-e7871ccc03b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:26.825243 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc79bc7-3e8a-4bba-9519-e7871ccc03b1 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:26.825531 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc79bc7-3e8a-4bba-9519-e7871ccc03b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:26.825811 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc79bc7-3e8a-4bba-9519-e7871ccc03b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:26.890855 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc79bc7-3e8a-4bba-9519-e7871ccc03b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:08:26.892303 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc79bc7-3e8a-4bba-9519-e7871ccc03b1 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:08:26.981495 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:08:26.988903 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 592/1198] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:08:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:08:26.994984 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4b3fea-dc5f-4f57-905b-dd9e60c6e6af None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:27.038480 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:08:27.095209 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4b3fea-dc5f-4f57-905b-dd9e60c6e6af None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:27.127791 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f96b2b-5d9b-4abf-889e-9d856a3a02dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:27.128643 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f96b2b-5d9b-4abf-889e-9d856a3a02dd None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:27.128924 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f96b2b-5d9b-4abf-889e-9d856a3a02dd None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:27.129237 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f96b2b-5d9b-4abf-889e-9d856a3a02dd None admin] PATH_INFO: `/v3/projects/4d5651ce77484ab9beca753ab22dfbac` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:27.130408 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f96b2b-5d9b-4abf-889e-9d856a3a02dd None admin] RBAC: Authorizing `identity:get_project(project_id=4d5651ce77484ab9beca753ab22dfbac)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:08:27.131818 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f96b2b-5d9b-4abf-889e-9d856a3a02dd None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:08:27.134013 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 607/1199] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 4 09:08:26 2022] GET /identity/v3/projects/4d5651ce77484ab9beca753ab22dfbac => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:08:27.140152 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc79bc7-3e8a-4bba-9519-e7871ccc03b1 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:27.242136 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc79bc7-3e8a-4bba-9519-e7871ccc03b1 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:27.275412 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4fbd24-5b79-4088-9507-3be66d5c61d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:27.276060 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4fbd24-5b79-4088-9507-3be66d5c61d9 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:27.276345 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4fbd24-5b79-4088-9507-3be66d5c61d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:27.276623 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4fbd24-5b79-4088-9507-3be66d5c61d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:27.277504 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4fbd24-5b79-4088-9507-3be66d5c61d9 None admin] RBAC: Adding query filter params (role.id=f36e053b442540f08041ac7878760f65, scope.project.id=4d5651ce77484ab9beca753ab22dfbac, user.id=85d261cffc4746e7a4815c14cdcca9ed) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:08:27.277816 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4fbd24-5b79-4088-9507-3be66d5c61d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:08:27.279246 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4fbd24-5b79-4088-9507-3be66d5c61d9 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:08:27.284873 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 593/1200] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 4 09:08:27 2022] GET /identity/v3/role_assignments?user.id=85d261cffc4746e7a4815c14cdcca9ed&scope.project.id=4d5651ce77484ab9beca753ab22dfbac&role.id=f36e053b442540f08041ac7878760f65 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:08:37.123735 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=45 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:08:37.282311 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=71 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:22.894944 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc19d9-4590-4708-9dc3-2096cdc1a0fb None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:22.894944 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc19d9-4590-4708-9dc3-2096cdc1a0fb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:22.898722 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc19d9-4590-4708-9dc3-2096cdc1a0fb None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:22.898722 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 608/1201] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:09:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:09:22.906239 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dcc2f6-bcfb-4c95-b380-f6764a890c42 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:22.907548 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dcc2f6-bcfb-4c95-b380-f6764a890c42 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:22.908444 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dcc2f6-bcfb-4c95-b380-f6764a890c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:22.942507 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-f9dcc2f6-bcfb-4c95-b380-f6764a890c42 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:09:22.945352 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f9dcc2f6-bcfb-4c95-b380-f6764a890c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:09:23.037194 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 594/1202] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:09:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:09:23.064861 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dc19d9-4590-4708-9dc3-2096cdc1a0fb None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:23.228021 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dc19d9-4590-4708-9dc3-2096cdc1a0fb None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:23.269245 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0394837f-e8dc-485c-a8ad-1b53b95c78ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:23.270168 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0394837f-e8dc-485c-a8ad-1b53b95c78ac None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:23.270432 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0394837f-e8dc-485c-a8ad-1b53b95c78ac None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:23.270735 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0394837f-e8dc-485c-a8ad-1b53b95c78ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:23.290096 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0394837f-e8dc-485c-a8ad-1b53b95c78ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:23.293166 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0394837f-e8dc-485c-a8ad-1b53b95c78ac None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:23.315638 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-0394837f-e8dc-485c-a8ad-1b53b95c78ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:09:23.315638 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:09:23.315638 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:09:23.315638 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:09:23.315638 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:09:23.315638 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:09:23.315638 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:09:23.315638 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:09:23.315638 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:09:23.315638 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:09:23.315638 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:09:23.315638 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:09:23.315638 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:09:23.315638 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:09:23.315638 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:09:23.317918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:09:23.317918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:09:23.317918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:09:23.317918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:09:23.317918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:09:23.317918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:09:23.317918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:09:23.317918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:09:23.317918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:09:23.317918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:09:23.317918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:09:23.317918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:09:23.317918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:09:23.317918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:09:23.317918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:09:23.317918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:09:23.317918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:09:23.320493 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:09:23.320493 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:09:23.320493 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:09:23.320493 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:09:23.323857 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 609/1203] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:09:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:09:23.325514 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dcc2f6-bcfb-4c95-b380-f6764a890c42 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:23.477189 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dcc2f6-bcfb-4c95-b380-f6764a890c42 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:23.517186 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:23.526514 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b884365f-13b1-466f-b574-aaf5564ecb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:23.527218 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b884365f-13b1-466f-b574-aaf5564ecb8c None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:23.527507 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b884365f-13b1-466f-b574-aaf5564ecb8c None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:23.527793 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b884365f-13b1-466f-b574-aaf5564ecb8c None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:23.528521 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b884365f-13b1-466f-b574-aaf5564ecb8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:09:23.528842 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b884365f-13b1-466f-b574-aaf5564ecb8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:23.530255 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b884365f-13b1-466f-b574-aaf5564ecb8c None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:23.544577 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 595/1204] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:09:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:09:23.550458 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0394837f-e8dc-485c-a8ad-1b53b95c78ac None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:23.626721 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:23.692662 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0394837f-e8dc-485c-a8ad-1b53b95c78ac None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:23.739109 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48ef100-241a-4230-bbc1-98b2192edffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:23.740196 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ef100-241a-4230-bbc1-98b2192edffb None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:23.740843 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ef100-241a-4230-bbc1-98b2192edffb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:23.741851 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ef100-241a-4230-bbc1-98b2192edffb None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:23.743013 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48ef100-241a-4230-bbc1-98b2192edffb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:23.744922 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48ef100-241a-4230-bbc1-98b2192edffb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:23.781163 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 610/1205] 10.222.0.71 () {60 vars in 1132 bytes} [Mon Jul 4 09:09:23 2022] POST /identity/v3/users => generated 251 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:09:25.997815 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b330fc-14a5-4626-b526-f0b1bad5b83f None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:25.998344 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b330fc-14a5-4626-b526-f0b1bad5b83f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:25.998446 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b330fc-14a5-4626-b526-f0b1bad5b83f None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:25.999746 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 596/1206] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:09:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:09:26.005084 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f92d6e-22ab-486a-925e-c1eb84e584b4 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:26.011668 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f92d6e-22ab-486a-925e-c1eb84e584b4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:26.011668 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f92d6e-22ab-486a-925e-c1eb84e584b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:26.024225 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-60f92d6e-22ab-486a-925e-c1eb84e584b4 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:09:26.025870 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-60f92d6e-22ab-486a-925e-c1eb84e584b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:09:26.114440 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 611/1207] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:09:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:09:26.140174 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b330fc-14a5-4626-b526-f0b1bad5b83f None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:26.277696 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b330fc-14a5-4626-b526-f0b1bad5b83f None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:26.325026 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167d9cf8-e31e-445d-924a-b28f2aa855f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:26.325854 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d9cf8-e31e-445d-924a-b28f2aa855f4 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:26.326130 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d9cf8-e31e-445d-924a-b28f2aa855f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:26.326410 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d9cf8-e31e-445d-924a-b28f2aa855f4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:26.341740 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167d9cf8-e31e-445d-924a-b28f2aa855f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:09:26.350592 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-167d9cf8-e31e-445d-924a-b28f2aa855f4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:09:26.350592 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:09:26.350592 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:09:26.350592 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:09:26.350592 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:09:26.350592 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:09:26.350592 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:09:26.350592 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:09:26.350592 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:09:26.350592 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:09:26.350592 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:09:26.350592 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:09:26.350592 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:09:26.350592 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:09:26.350592 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:09:26.350592 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:09:26.351604 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:09:26.351604 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:09:26.351604 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:09:26.351604 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:09:26.351604 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:09:26.351604 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:09:26.351604 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:09:26.351604 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:09:26.351604 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:09:26.351604 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:09:26.351604 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:09:26.351604 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:09:26.351604 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:09:26.351604 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:09:26.351604 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:09:26.351604 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:09:26.352470 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:09:26.352470 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:09:26.352470 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:09:26.352470 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:09:26.352470 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:09:26.352470 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:09:26.352470 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:09:26.352470 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:09:26.352470 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:09:26.352470 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:09:26.352470 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:09:26.352470 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:09:26.352470 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 597/1208] 10.222.0.71 () {56 vars in 1102 bytes} [Mon Jul 4 09:09:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:09:26.360290 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f92d6e-22ab-486a-925e-c1eb84e584b4 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:26.413292 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:26.486979 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f92d6e-22ab-486a-925e-c1eb84e584b4 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:26.531251 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e295866-17be-4cbc-8ab3-ea0a5bb2b07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:26.531857 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e295866-17be-4cbc-8ab3-ea0a5bb2b07d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:26.532164 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e295866-17be-4cbc-8ab3-ea0a5bb2b07d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:26.532515 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e295866-17be-4cbc-8ab3-ea0a5bb2b07d None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:26.533579 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e295866-17be-4cbc-8ab3-ea0a5bb2b07d None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:09:26.533881 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e295866-17be-4cbc-8ab3-ea0a5bb2b07d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:26.535697 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e295866-17be-4cbc-8ab3-ea0a5bb2b07d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:26.547450 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 612/1209] 10.222.0.71 () {56 vars in 1103 bytes} [Mon Jul 4 09:09:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:09:26.553192 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167d9cf8-e31e-445d-924a-b28f2aa855f4 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:26.633377 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:26.692377 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167d9cf8-e31e-445d-924a-b28f2aa855f4 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:26.736530 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9bc400-ebc0-4d6c-b417-bed0aae20e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:26.737596 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9bc400-ebc0-4d6c-b417-bed0aae20e2d None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:26.738021 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9bc400-ebc0-4d6c-b417-bed0aae20e2d None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:26.738345 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9bc400-ebc0-4d6c-b417-bed0aae20e2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:26.751333 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9bc400-ebc0-4d6c-b417-bed0aae20e2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:26.753335 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9bc400-ebc0-4d6c-b417-bed0aae20e2d None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:26.786718 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-ab9bc400-ebc0-4d6c-b417-bed0aae20e2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:09:26.786718 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:09:26.786718 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:09:26.786718 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:09:26.786718 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:09:26.786718 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:09:26.786718 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:09:26.786718 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:09:26.786718 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:09:26.786718 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:09:26.786718 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:09:26.786718 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:09:26.786718 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:09:26.786718 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:09:26.786718 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:09:26.788634 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:09:26.788634 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:09:26.788634 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:09:26.788634 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:09:26.788634 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:09:26.788634 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:09:26.788634 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:09:26.788634 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:09:26.788634 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:09:26.788634 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:09:26.788634 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:09:26.788634 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:09:26.788634 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:09:26.788634 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:09:26.788634 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:09:26.788634 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:09:26.788634 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:09:26.789695 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:09:26.789695 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:09:26.789695 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:09:26.789695 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:09:26.789695 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 598/1210] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:09:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:09:26.795345 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e295866-17be-4cbc-8ab3-ea0a5bb2b07d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:26.919548 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:26.923949 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e295866-17be-4cbc-8ab3-ea0a5bb2b07d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:26.974543 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19262be-b519-485c-ba78-b852b8644d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:26.975716 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19262be-b519-485c-ba78-b852b8644d08 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:26.976223 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19262be-b519-485c-ba78-b852b8644d08 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:26.976514 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19262be-b519-485c-ba78-b852b8644d08 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:26.977460 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19262be-b519-485c-ba78-b852b8644d08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:09:26.977778 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19262be-b519-485c-ba78-b852b8644d08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:26.979591 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19262be-b519-485c-ba78-b852b8644d08 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:26.995049 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 613/1211] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:09:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:09:27.003978 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9bc400-ebc0-4d6c-b417-bed0aae20e2d None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:27.128609 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9bc400-ebc0-4d6c-b417-bed0aae20e2d None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:27.129963 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:27.169472 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0b28d3-711a-4a74-ab6b-ec13ed5d0a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:27.170375 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0b28d3-711a-4a74-ab6b-ec13ed5d0a27 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:27.170667 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0b28d3-711a-4a74-ab6b-ec13ed5d0a27 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:27.170941 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0b28d3-711a-4a74-ab6b-ec13ed5d0a27 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:27.179028 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0b28d3-711a-4a74-ab6b-ec13ed5d0a27 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:27.180487 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0b28d3-711a-4a74-ab6b-ec13ed5d0a27 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:27.198037 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-ef0b28d3-711a-4a74-ab6b-ec13ed5d0a27 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 04 09:09:27.198037 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:09:27.198037 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:09:27.198037 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:09:27.198037 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:09:27.198037 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:09:27.198037 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:09:27.198037 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:09:27.198037 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:09:27.198037 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:09:27.198037 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:09:27.198037 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:09:27.198037 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:09:27.198037 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:09:27.198037 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:09:27.198037 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:09:27.199727 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:09:27.199727 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:09:27.199727 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:09:27.199727 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:09:27.199727 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:09:27.199727 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:09:27.199727 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:09:27.199727 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:09:27.199727 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:09:27.199727 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:09:27.199727 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:09:27.199727 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:09:27.199727 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:09:27.199727 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:09:27.199727 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:09:27.199727 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:09:27.199727 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:09:27.199727 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:09:27.201585 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:09:27.201585 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:09:27.201585 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:09:27.201585 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:09:27.201585 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:09:27.201585 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:09:27.201585 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:09:27.201585 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:09:27.201585 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 04 09:09:27.201585 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:09:27.201585 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 599/1212] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:09:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:09:27.209244 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19262be-b519-485c-ba78-b852b8644d08 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:27.331463 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19262be-b519-485c-ba78-b852b8644d08 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:27.373127 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51feb781-fc5a-43c9-9a69-1e20c71ba321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:27.373803 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51feb781-fc5a-43c9-9a69-1e20c71ba321 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:27.374088 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51feb781-fc5a-43c9-9a69-1e20c71ba321 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:27.374391 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51feb781-fc5a-43c9-9a69-1e20c71ba321 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:27.375326 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51feb781-fc5a-43c9-9a69-1e20c71ba321 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:09:27.375643 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51feb781-fc5a-43c9-9a69-1e20c71ba321 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:27.377083 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51feb781-fc5a-43c9-9a69-1e20c71ba321 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:27.397568 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 614/1213] 10.222.0.71 () {56 vars in 1143 bytes} [Mon Jul 4 09:09:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:09:27.403681 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0b28d3-711a-4a74-ab6b-ec13ed5d0a27 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:27.529154 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0b28d3-711a-4a74-ab6b-ec13ed5d0a27 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:27.577433 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0386c72c-bcad-4e4b-9b93-6da888598c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:27.578128 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386c72c-bcad-4e4b-9b93-6da888598c85 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:27.578420 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386c72c-bcad-4e4b-9b93-6da888598c85 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:27.578709 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386c72c-bcad-4e4b-9b93-6da888598c85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:27.612088 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:27.659349 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0386c72c-bcad-4e4b-9b93-6da888598c85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:27.660944 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0386c72c-bcad-4e4b-9b93-6da888598c85 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:27.788794 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 600/1214] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:09:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:09:27.795472 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51feb781-fc5a-43c9-9a69-1e20c71ba321 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:27.817918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:27.914945 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51feb781-fc5a-43c9-9a69-1e20c71ba321 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:27.955831 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ea50bc-9c69-4762-bd6e-6a11d362e3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:27.956636 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea50bc-9c69-4762-bd6e-6a11d362e3ae None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:27.956913 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea50bc-9c69-4762-bd6e-6a11d362e3ae None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:27.957216 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea50bc-9c69-4762-bd6e-6a11d362e3ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:27.969176 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ea50bc-9c69-4762-bd6e-6a11d362e3ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:27.970642 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ea50bc-9c69-4762-bd6e-6a11d362e3ae None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:27.990416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-36ea50bc-9c69-4762-bd6e-6a11d362e3ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:09:27.990416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:09:27.990416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:09:27.990416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:09:27.990416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:09:27.990416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:09:27.990416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:09:27.990416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:09:27.990416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:09:27.990416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:09:27.990416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:09:27.990416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:09:27.990416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:09:27.990416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:09:27.990416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:09:27.992274 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:09:27.992274 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:09:27.992274 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:09:27.992274 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:09:27.992274 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:09:27.992274 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:09:27.992274 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:09:27.992274 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:09:27.992274 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:09:27.992274 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:09:27.992274 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:09:27.992274 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:09:27.992274 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:09:27.992274 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:09:27.992274 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:09:27.992274 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:09:27.992274 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:09:27.993431 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:09:27.993431 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:09:27.993431 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:09:27.993431 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:09:27.993431 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 615/1215] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:09:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:09:28.000581 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0386c72c-bcad-4e4b-9b93-6da888598c85 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:28.068967 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:28.115398 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0386c72c-bcad-4e4b-9b93-6da888598c85 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:28.152836 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f142b40-c308-4282-b6fa-833ed579afd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:28.153536 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f142b40-c308-4282-b6fa-833ed579afd1 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:28.153818 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f142b40-c308-4282-b6fa-833ed579afd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:28.154094 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f142b40-c308-4282-b6fa-833ed579afd1 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:28.154813 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f142b40-c308-4282-b6fa-833ed579afd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:09:28.155121 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f142b40-c308-4282-b6fa-833ed579afd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:28.156484 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f142b40-c308-4282-b6fa-833ed579afd1 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:28.173474 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 601/1216] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:09:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:09:28.174239 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ea50bc-9c69-4762-bd6e-6a11d362e3ae None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:28.260025 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:28.297502 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ea50bc-9c69-4762-bd6e-6a11d362e3ae None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:28.337196 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663bd596-d24b-4dfa-bed5-c61ad41bf86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:28.337964 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663bd596-d24b-4dfa-bed5-c61ad41bf86c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:28.338249 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663bd596-d24b-4dfa-bed5-c61ad41bf86c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:28.338596 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663bd596-d24b-4dfa-bed5-c61ad41bf86c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:28.345791 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663bd596-d24b-4dfa-bed5-c61ad41bf86c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:28.347275 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663bd596-d24b-4dfa-bed5-c61ad41bf86c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:28.361867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-663bd596-d24b-4dfa-bed5-c61ad41bf86c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:09:28.361867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:09:28.361867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:09:28.361867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:09:28.361867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:09:28.361867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:09:28.361867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:09:28.361867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:09:28.361867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:09:28.361867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:09:28.361867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:09:28.361867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:09:28.361867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:09:28.361867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:09:28.361867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:09:28.362797 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:09:28.362797 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:09:28.362797 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:09:28.362797 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:09:28.362797 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:09:28.362797 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:09:28.362797 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:09:28.362797 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:09:28.362797 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:09:28.362797 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:09:28.362797 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:09:28.362797 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:09:28.362797 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:09:28.362797 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:09:28.362797 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:09:28.362797 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:09:28.362797 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:09:28.363701 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:09:28.363701 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:09:28.363701 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:09:28.363701 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:09:28.363701 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:09:28.363701 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:09:28.363701 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:09:28.363701 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:09:28.363701 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 616/1217] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 4 09:09:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:09:28.370373 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f142b40-c308-4282-b6fa-833ed579afd1 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:28.484952 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f142b40-c308-4282-b6fa-833ed579afd1 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:28.512012 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:28.527240 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e47f31-8077-4be8-ae0a-cc0422414e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:28.527913 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e47f31-8077-4be8-ae0a-cc0422414e1d None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:28.528193 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e47f31-8077-4be8-ae0a-cc0422414e1d None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:28.528475 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e47f31-8077-4be8-ae0a-cc0422414e1d None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:28.529298 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e47f31-8077-4be8-ae0a-cc0422414e1d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:09:28.529615 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e47f31-8077-4be8-ae0a-cc0422414e1d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:28.531067 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e47f31-8077-4be8-ae0a-cc0422414e1d None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:28.544590 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 602/1218] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 4 09:09:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:09:28.550777 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663bd596-d24b-4dfa-bed5-c61ad41bf86c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:28.665110 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663bd596-d24b-4dfa-bed5-c61ad41bf86c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:28.696366 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:28.704027 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a7fbc3-6b9f-4cd7-966e-9fef9bebe840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:28.704848 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7fbc3-6b9f-4cd7-966e-9fef9bebe840 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:28.705244 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7fbc3-6b9f-4cd7-966e-9fef9bebe840 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:28.705585 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7fbc3-6b9f-4cd7-966e-9fef9bebe840 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:28.706542 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a7fbc3-6b9f-4cd7-966e-9fef9bebe840 None admin] RBAC: Adding query filter params (role.id=5752a9bdfe4b482ab37836eba7bd536b, scope.project.id=4076e316243946e980ce47386b18b28f, user.id=293c695f15c8437d81a619f840318f44) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:09:28.707014 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a7fbc3-6b9f-4cd7-966e-9fef9bebe840 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:28.708616 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a7fbc3-6b9f-4cd7-966e-9fef9bebe840 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:28.725074 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 617/1219] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 4 09:09:28 2022] GET /identity/v3/role_assignments?user.id=293c695f15c8437d81a619f840318f44&scope.project.id=4076e316243946e980ce47386b18b28f&role.id=5752a9bdfe4b482ab37836eba7bd536b => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:09:30.941708 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10934d1-11ca-4d0b-9d81-edffee14a10f None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:30.942297 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10934d1-11ca-4d0b-9d81-edffee14a10f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:30.942297 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10934d1-11ca-4d0b-9d81-edffee14a10f None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:30.943660 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 603/1220] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:09:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:09:30.949080 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629dc6ce-4504-450b-afd8-13ad6a51b5a2 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:30.949488 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629dc6ce-4504-450b-afd8-13ad6a51b5a2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:30.949669 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629dc6ce-4504-450b-afd8-13ad6a51b5a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:30.966528 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-629dc6ce-4504-450b-afd8-13ad6a51b5a2 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:09:30.967724 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-629dc6ce-4504-450b-afd8-13ad6a51b5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:09:31.042885 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 618/1221] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:09:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:09:31.070646 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10934d1-11ca-4d0b-9d81-edffee14a10f None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:31.191116 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10934d1-11ca-4d0b-9d81-edffee14a10f None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:31.229933 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d28cef-3b8b-445e-89c4-7a37296b0987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:31.230736 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d28cef-3b8b-445e-89c4-7a37296b0987 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:31.231055 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d28cef-3b8b-445e-89c4-7a37296b0987 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:31.231335 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d28cef-3b8b-445e-89c4-7a37296b0987 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:31.244444 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d28cef-3b8b-445e-89c4-7a37296b0987 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:09:31.255156 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-59d28cef-3b8b-445e-89c4-7a37296b0987 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:09:31.255156 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:09:31.255156 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:09:31.255156 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:09:31.255156 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:09:31.255156 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:09:31.255156 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:09:31.255156 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:09:31.255156 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:09:31.255156 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:09:31.255156 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:09:31.255156 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:09:31.255156 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:09:31.255156 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:09:31.255156 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:09:31.255156 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:09:31.256366 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:09:31.256366 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:09:31.256366 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:09:31.256366 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:09:31.256366 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:09:31.256366 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:09:31.256366 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:09:31.256366 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:09:31.256366 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:09:31.256366 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:09:31.256366 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:09:31.256366 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:09:31.256366 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:09:31.256366 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:09:31.256366 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:09:31.256366 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:09:31.257337 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:09:31.257337 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:09:31.257337 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:09:31.257337 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:09:31.257337 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:09:31.257337 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:09:31.257337 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:09:31.257337 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:09:31.257337 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:09:31.257337 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:09:31.257337 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:09:31.257337 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:09:31.257337 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 604/1222] 10.222.0.71 () {56 vars in 1102 bytes} [Mon Jul 4 09:09:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:09:31.263545 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629dc6ce-4504-450b-afd8-13ad6a51b5a2 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:31.365498 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629dc6ce-4504-450b-afd8-13ad6a51b5a2 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:31.390329 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:31.402634 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b38169-7f75-4416-8119-481b26bede39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:31.403318 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b38169-7f75-4416-8119-481b26bede39 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:31.403608 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b38169-7f75-4416-8119-481b26bede39 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:31.403874 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b38169-7f75-4416-8119-481b26bede39 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:31.404622 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b38169-7f75-4416-8119-481b26bede39 None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:09:31.404933 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b38169-7f75-4416-8119-481b26bede39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:31.406326 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b38169-7f75-4416-8119-481b26bede39 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:31.415745 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 619/1223] 10.222.0.71 () {56 vars in 1103 bytes} [Mon Jul 4 09:09:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:09:31.421920 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d28cef-3b8b-445e-89c4-7a37296b0987 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:31.465524 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:31.526947 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d28cef-3b8b-445e-89c4-7a37296b0987 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:31.562736 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176d9417-4fbd-4ba3-bf7c-4b3e12fdb34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:31.563534 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d9417-4fbd-4ba3-bf7c-4b3e12fdb34f None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:31.563841 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d9417-4fbd-4ba3-bf7c-4b3e12fdb34f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:31.564101 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d9417-4fbd-4ba3-bf7c-4b3e12fdb34f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:31.575018 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176d9417-4fbd-4ba3-bf7c-4b3e12fdb34f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:31.576498 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176d9417-4fbd-4ba3-bf7c-4b3e12fdb34f None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:31.594878 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-176d9417-4fbd-4ba3-bf7c-4b3e12fdb34f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:09:31.594878 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:09:31.594878 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:09:31.594878 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:09:31.594878 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:09:31.594878 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:09:31.594878 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:09:31.594878 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:09:31.594878 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:09:31.594878 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:09:31.594878 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:09:31.594878 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:09:31.594878 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:09:31.594878 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:09:31.594878 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:09:31.596513 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:09:31.596513 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:09:31.596513 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:09:31.596513 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:09:31.596513 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:09:31.596513 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:09:31.596513 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:09:31.596513 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:09:31.596513 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:09:31.596513 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:09:31.596513 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:09:31.596513 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:09:31.596513 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:09:31.596513 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:09:31.596513 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:09:31.596513 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:09:31.596513 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:09:31.598314 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:09:31.598314 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:09:31.598314 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:09:31.598314 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:09:31.598314 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 605/1224] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:09:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:09:31.603215 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b38169-7f75-4416-8119-481b26bede39 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:31.716989 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b38169-7f75-4416-8119-481b26bede39 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:31.755557 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3192080f-0f14-4e45-bf55-49ef9ef1e649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:31.756210 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192080f-0f14-4e45-bf55-49ef9ef1e649 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:31.756987 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192080f-0f14-4e45-bf55-49ef9ef1e649 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:31.757299 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192080f-0f14-4e45-bf55-49ef9ef1e649 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:31.758022 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3192080f-0f14-4e45-bf55-49ef9ef1e649 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:09:31.758329 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3192080f-0f14-4e45-bf55-49ef9ef1e649 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:31.759697 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3192080f-0f14-4e45-bf55-49ef9ef1e649 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:31.771620 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 620/1225] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:09:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:09:31.778201 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176d9417-4fbd-4ba3-bf7c-4b3e12fdb34f None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:31.883466 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:31.890829 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176d9417-4fbd-4ba3-bf7c-4b3e12fdb34f None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:31.928792 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80860bd-e941-4883-b825-6af623a55054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:31.929660 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80860bd-e941-4883-b825-6af623a55054 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:31.929951 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80860bd-e941-4883-b825-6af623a55054 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:31.930283 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80860bd-e941-4883-b825-6af623a55054 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:31.939389 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80860bd-e941-4883-b825-6af623a55054 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:31.940887 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80860bd-e941-4883-b825-6af623a55054 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:31.956618 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-c80860bd-e941-4883-b825-6af623a55054 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 04 09:09:31.956618 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:09:31.956618 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:09:31.956618 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:09:31.956618 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:09:31.956618 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:09:31.956618 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:09:31.956618 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:09:31.956618 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:09:31.956618 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:09:31.956618 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:09:31.956618 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:09:31.956618 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:09:31.956618 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:09:31.956618 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:09:31.956618 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:09:31.963945 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:09:31.963945 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:09:31.963945 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:09:31.963945 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:09:31.963945 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:09:31.963945 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:09:31.963945 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:09:31.963945 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:09:31.963945 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:09:31.963945 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:09:31.963945 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:09:31.963945 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:09:31.963945 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:09:31.963945 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:09:31.963945 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:09:31.963945 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:09:31.963945 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:09:31.963945 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:09:31.965399 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:09:31.965399 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:09:31.965399 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:09:31.965399 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:09:31.965399 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:09:31.965399 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:09:31.965399 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:09:31.965399 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:09:31.965399 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 04 09:09:31.965399 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:09:31.965399 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 606/1226] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:09:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:09:31.966130 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3192080f-0f14-4e45-bf55-49ef9ef1e649 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:32.018209 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:32.076384 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3192080f-0f14-4e45-bf55-49ef9ef1e649 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:32.118435 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02570b69-17e4-429e-9bd7-3a6a6d8c4ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:32.119390 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02570b69-17e4-429e-9bd7-3a6a6d8c4ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:32.119858 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02570b69-17e4-429e-9bd7-3a6a6d8c4ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:32.120191 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02570b69-17e4-429e-9bd7-3a6a6d8c4ce3 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:32.121106 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02570b69-17e4-429e-9bd7-3a6a6d8c4ce3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:09:32.121454 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02570b69-17e4-429e-9bd7-3a6a6d8c4ce3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:32.123198 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02570b69-17e4-429e-9bd7-3a6a6d8c4ce3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:32.143001 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 621/1227] 10.222.0.71 () {56 vars in 1143 bytes} [Mon Jul 4 09:09:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:09:32.148916 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80860bd-e941-4883-b825-6af623a55054 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:32.259410 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80860bd-e941-4883-b825-6af623a55054 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:32.295346 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75741950-5edb-4e8f-89e2-292e9a05aa9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:32.296143 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75741950-5edb-4e8f-89e2-292e9a05aa9a None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:32.296434 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75741950-5edb-4e8f-89e2-292e9a05aa9a None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:32.296725 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75741950-5edb-4e8f-89e2-292e9a05aa9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:32.309469 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75741950-5edb-4e8f-89e2-292e9a05aa9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:32.310938 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75741950-5edb-4e8f-89e2-292e9a05aa9a None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:32.329808 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-75741950-5edb-4e8f-89e2-292e9a05aa9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:09:32.329808 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:09:32.329808 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:09:32.329808 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:09:32.329808 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:09:32.329808 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:09:32.329808 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:09:32.329808 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:09:32.329808 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:09:32.329808 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:09:32.329808 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:09:32.329808 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:09:32.329808 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:09:32.329808 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:09:32.329808 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:09:32.331057 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:09:32.331057 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:09:32.331057 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:09:32.331057 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:09:32.331057 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:09:32.331057 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:09:32.331057 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:09:32.331057 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:09:32.331057 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:09:32.331057 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:09:32.331057 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:09:32.331057 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:09:32.331057 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:09:32.331057 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:09:32.331057 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:09:32.331057 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:09:32.331057 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:09:32.332201 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:09:32.332201 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:09:32.332201 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:09:32.332201 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:09:32.332201 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 607/1228] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:09:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:09:32.337330 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02570b69-17e4-429e-9bd7-3a6a6d8c4ce3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:32.454399 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02570b69-17e4-429e-9bd7-3a6a6d8c4ce3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:32.456740 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:32.491797 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93e0758-d86b-43a4-94f7-660d45b38857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:32.492471 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e0758-d86b-43a4-94f7-660d45b38857 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:32.492784 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e0758-d86b-43a4-94f7-660d45b38857 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:32.493028 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e0758-d86b-43a4-94f7-660d45b38857 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:32.493815 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93e0758-d86b-43a4-94f7-660d45b38857 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:09:32.494089 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93e0758-d86b-43a4-94f7-660d45b38857 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:32.495451 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93e0758-d86b-43a4-94f7-660d45b38857 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:32.506979 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 622/1229] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:09:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:09:32.512808 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75741950-5edb-4e8f-89e2-292e9a05aa9a None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:32.522318 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:32.619459 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75741950-5edb-4e8f-89e2-292e9a05aa9a None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:32.653971 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015ef1b9-337d-40f6-b892-167f0aca765f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:32.654779 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015ef1b9-337d-40f6-b892-167f0aca765f None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:32.655135 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015ef1b9-337d-40f6-b892-167f0aca765f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:32.655336 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015ef1b9-337d-40f6-b892-167f0aca765f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:32.663250 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015ef1b9-337d-40f6-b892-167f0aca765f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:32.664841 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015ef1b9-337d-40f6-b892-167f0aca765f None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:32.681656 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-015ef1b9-337d-40f6-b892-167f0aca765f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:09:32.681656 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:09:32.681656 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:09:32.681656 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:09:32.681656 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:09:32.681656 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:09:32.681656 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:09:32.681656 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:09:32.681656 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:09:32.681656 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:09:32.681656 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:09:32.681656 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:09:32.681656 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:09:32.681656 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:09:32.681656 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:09:32.682820 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:09:32.682820 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:09:32.682820 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:09:32.682820 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:09:32.682820 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:09:32.682820 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:09:32.682820 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:09:32.682820 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:09:32.682820 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:09:32.682820 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:09:32.682820 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:09:32.682820 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:09:32.682820 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:09:32.682820 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:09:32.682820 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:09:32.682820 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:09:32.682820 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:09:32.688763 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:09:32.688763 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:09:32.688763 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:09:32.688763 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:09:32.688763 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:09:32.688763 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:09:32.688763 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:09:32.688763 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:09:32.688763 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 608/1230] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 4 09:09:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:09:32.689413 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93e0758-d86b-43a4-94f7-660d45b38857 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:32.804638 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93e0758-d86b-43a4-94f7-660d45b38857 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:32.841976 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997d5efa-5d53-40c1-9bad-f891e18e4722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:32.842689 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997d5efa-5d53-40c1-9bad-f891e18e4722 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:32.843035 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997d5efa-5d53-40c1-9bad-f891e18e4722 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:32.843301 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997d5efa-5d53-40c1-9bad-f891e18e4722 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:32.844175 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997d5efa-5d53-40c1-9bad-f891e18e4722 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:09:32.844542 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997d5efa-5d53-40c1-9bad-f891e18e4722 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:32.846473 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997d5efa-5d53-40c1-9bad-f891e18e4722 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:32.859881 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 623/1231] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 4 09:09:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:09:32.867575 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015ef1b9-337d-40f6-b892-167f0aca765f None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:32.940029 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:32.982434 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015ef1b9-337d-40f6-b892-167f0aca765f None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:33.023802 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6000ea-60cd-48a3-9372-ab72b26a5dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:33.025494 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6000ea-60cd-48a3-9372-ab72b26a5dc3 None admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:33.025925 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6000ea-60cd-48a3-9372-ab72b26a5dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:33.026214 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6000ea-60cd-48a3-9372-ab72b26a5dc3 None admin] PATH_INFO: `/v3/projects/4076e316243946e980ce47386b18b28f/users/293c695f15c8437d81a619f840318f44/roles/5752a9bdfe4b482ab37836eba7bd536b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:33.044397 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6000ea-60cd-48a3-9372-ab72b26a5dc3 None admin] RBAC: Authorizing `identity:create_grant(project_id=4076e316243946e980ce47386b18b28f, user_id=293c695f15c8437d81a619f840318f44, role_id=5752a9bdfe4b482ab37836eba7bd536b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:33.047153 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6000ea-60cd-48a3-9372-ab72b26a5dc3 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:33.063824 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 609/1232] 10.222.0.71 () {58 vars in 1439 bytes} [Mon Jul 4 09:09:32 2022] PUT /identity/v3/projects/4076e316243946e980ce47386b18b28f/users/293c695f15c8437d81a619f840318f44/roles/5752a9bdfe4b482ab37836eba7bd536b => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:09:35.140707 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36639f62-f150-4389-9e21-17d7a1b41906 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:35.141494 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36639f62-f150-4389-9e21-17d7a1b41906 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:35.141494 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36639f62-f150-4389-9e21-17d7a1b41906 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:35.142848 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 624/1233] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:09:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:09:35.149120 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7946e70-d4f7-4726-8397-0d94aa8bec13 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:35.149722 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7946e70-d4f7-4726-8397-0d94aa8bec13 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:35.149722 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7946e70-d4f7-4726-8397-0d94aa8bec13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:35.168136 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-e7946e70-d4f7-4726-8397-0d94aa8bec13 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:09:35.169647 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e7946e70-d4f7-4726-8397-0d94aa8bec13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:09:35.249328 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 610/1234] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:09:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:09:35.276618 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36639f62-f150-4389-9e21-17d7a1b41906 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:35.305910 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:35.392162 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36639f62-f150-4389-9e21-17d7a1b41906 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:35.430037 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929ebd3c-52fa-4bae-8f9c-a62dfd0e4928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:35.431004 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929ebd3c-52fa-4bae-8f9c-a62dfd0e4928 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:35.431368 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929ebd3c-52fa-4bae-8f9c-a62dfd0e4928 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:35.431731 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929ebd3c-52fa-4bae-8f9c-a62dfd0e4928 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:35.443768 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929ebd3c-52fa-4bae-8f9c-a62dfd0e4928 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:09:35.452449 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-929ebd3c-52fa-4bae-8f9c-a62dfd0e4928 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:09:35.452449 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:09:35.452449 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:09:35.452449 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:09:35.452449 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:09:35.452449 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:09:35.452449 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:09:35.452449 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:09:35.452449 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:09:35.452449 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:09:35.452449 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:09:35.452449 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:09:35.452449 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:09:35.452449 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:09:35.452449 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:09:35.452449 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:09:35.453645 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:09:35.453645 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:09:35.453645 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:09:35.453645 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:09:35.453645 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:09:35.453645 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:09:35.453645 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:09:35.453645 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:09:35.453645 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:09:35.453645 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:09:35.453645 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:09:35.453645 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:09:35.453645 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:09:35.453645 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:09:35.453645 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:09:35.453645 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:09:35.454675 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:09:35.454675 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:09:35.454675 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:09:35.454675 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:09:35.454675 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:09:35.454675 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:09:35.454675 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:09:35.454675 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:09:35.454675 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:09:35.454675 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:09:35.454675 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:09:35.454675 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:09:35.458641 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 625/1235] 10.222.0.71 () {56 vars in 1102 bytes} [Mon Jul 4 09:09:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:09:35.459756 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7946e70-d4f7-4726-8397-0d94aa8bec13 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:35.512665 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:35.580613 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7946e70-d4f7-4726-8397-0d94aa8bec13 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:35.639543 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d27e25-27f1-4524-881d-10a705651f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:35.640305 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d27e25-27f1-4524-881d-10a705651f75 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:35.640692 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d27e25-27f1-4524-881d-10a705651f75 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:35.641059 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d27e25-27f1-4524-881d-10a705651f75 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:35.642398 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d27e25-27f1-4524-881d-10a705651f75 None admin] RBAC: Adding query filter params (name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:09:35.642830 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d27e25-27f1-4524-881d-10a705651f75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:35.645090 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d27e25-27f1-4524-881d-10a705651f75 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:35.655298 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 611/1236] 10.222.0.71 () {56 vars in 1103 bytes} [Mon Jul 4 09:09:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:09:35.661557 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929ebd3c-52fa-4bae-8f9c-a62dfd0e4928 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:35.781921 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:35.800768 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929ebd3c-52fa-4bae-8f9c-a62dfd0e4928 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:35.840429 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f92e47-3019-4158-9a7c-bf224bd8c2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:35.841349 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f92e47-3019-4158-9a7c-bf224bd8c2ef None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:35.841699 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f92e47-3019-4158-9a7c-bf224bd8c2ef None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:35.842013 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f92e47-3019-4158-9a7c-bf224bd8c2ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:35.855232 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f92e47-3019-4158-9a7c-bf224bd8c2ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:35.857623 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f92e47-3019-4158-9a7c-bf224bd8c2ef None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:35.876980 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-76f92e47-3019-4158-9a7c-bf224bd8c2ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:09:35.876980 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:09:35.876980 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:09:35.876980 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:09:35.876980 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:09:35.876980 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:09:35.876980 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:09:35.876980 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:09:35.876980 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:09:35.876980 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:09:35.876980 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:09:35.876980 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:09:35.876980 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:09:35.876980 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:09:35.876980 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:09:35.878134 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:09:35.878134 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:09:35.878134 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:09:35.878134 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:09:35.878134 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:09:35.878134 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:09:35.878134 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:09:35.878134 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:09:35.878134 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:09:35.878134 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:09:35.878134 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:09:35.878134 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:09:35.878134 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:09:35.878134 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:09:35.878134 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:09:35.878134 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:09:35.878134 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:09:35.879222 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:09:35.879222 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:09:35.879222 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:09:35.879222 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:09:35.879222 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 626/1237] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:09:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:09:35.884411 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d27e25-27f1-4524-881d-10a705651f75 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:35.994848 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d27e25-27f1-4524-881d-10a705651f75 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:36.005929 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:36.034965 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5be456-eb9f-4315-afa1-496c70beeccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:36.035629 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5be456-eb9f-4315-afa1-496c70beeccf None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:36.035907 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5be456-eb9f-4315-afa1-496c70beeccf None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:36.036189 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5be456-eb9f-4315-afa1-496c70beeccf None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:36.036912 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5be456-eb9f-4315-afa1-496c70beeccf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:09:36.037272 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5be456-eb9f-4315-afa1-496c70beeccf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:36.040024 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5be456-eb9f-4315-afa1-496c70beeccf None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:36.052956 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 612/1238] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:09:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:09:36.058929 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f92e47-3019-4158-9a7c-bf224bd8c2ef None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:36.190174 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f92e47-3019-4158-9a7c-bf224bd8c2ef None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:36.234351 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e89295-f96e-4bce-b281-aa26cd1379a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:36.235330 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e89295-f96e-4bce-b281-aa26cd1379a3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:36.235726 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e89295-f96e-4bce-b281-aa26cd1379a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:36.236160 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e89295-f96e-4bce-b281-aa26cd1379a3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:36.240214 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:36.246491 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e89295-f96e-4bce-b281-aa26cd1379a3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:36.248291 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e89295-f96e-4bce-b281-aa26cd1379a3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:36.263805 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-35e89295-f96e-4bce-b281-aa26cd1379a3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 04 09:09:36.263805 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:09:36.263805 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:09:36.263805 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:09:36.263805 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:09:36.263805 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:09:36.263805 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:09:36.263805 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:09:36.263805 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:09:36.263805 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:09:36.263805 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:09:36.263805 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:09:36.263805 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:09:36.263805 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:09:36.263805 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:09:36.263805 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:09:36.264994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:09:36.264994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:09:36.264994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:09:36.264994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:09:36.264994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:09:36.264994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:09:36.264994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:09:36.264994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:09:36.264994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:09:36.264994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:09:36.264994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:09:36.264994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:09:36.264994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:09:36.264994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:09:36.264994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:09:36.264994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:09:36.264994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:09:36.264994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:09:36.266040 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:09:36.266040 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:09:36.266040 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:09:36.266040 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:09:36.266040 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:09:36.266040 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:09:36.266040 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:09:36.266040 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:09:36.266040 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 04 09:09:36.266040 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:09:36.266040 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 627/1239] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:09:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:09:36.270771 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5be456-eb9f-4315-afa1-496c70beeccf None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:36.384340 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5be456-eb9f-4315-afa1-496c70beeccf None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:36.423117 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eee183-df80-4d94-b1e1-602c913d2f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:36.423798 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eee183-df80-4d94-b1e1-602c913d2f74 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:36.424104 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eee183-df80-4d94-b1e1-602c913d2f74 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:36.424347 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eee183-df80-4d94-b1e1-602c913d2f74 None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:36.425213 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eee183-df80-4d94-b1e1-602c913d2f74 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:09:36.425558 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eee183-df80-4d94-b1e1-602c913d2f74 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:36.426945 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eee183-df80-4d94-b1e1-602c913d2f74 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:36.446453 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 613/1240] 10.222.0.71 () {56 vars in 1143 bytes} [Mon Jul 4 09:09:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:09:36.452234 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e89295-f96e-4bce-b281-aa26cd1379a3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:36.566428 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e89295-f96e-4bce-b281-aa26cd1379a3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:36.605235 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c5f0cd-d0a4-447c-95d1-bbb1c2bb15e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:36.605888 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5f0cd-d0a4-447c-95d1-bbb1c2bb15e2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:36.606174 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5f0cd-d0a4-447c-95d1-bbb1c2bb15e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:36.606458 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5f0cd-d0a4-447c-95d1-bbb1c2bb15e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:36.673553 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:36.681636 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c5f0cd-d0a4-447c-95d1-bbb1c2bb15e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:36.683217 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c5f0cd-d0a4-447c-95d1-bbb1c2bb15e2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:36.796891 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 628/1241] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:09:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:09:36.803012 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eee183-df80-4d94-b1e1-602c913d2f74 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:36.833147 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:36.909067 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eee183-df80-4d94-b1e1-602c913d2f74 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:36.946587 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7e5612-ca7b-469c-8452-74458dc6a530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:36.947451 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e5612-ca7b-469c-8452-74458dc6a530 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:36.947745 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e5612-ca7b-469c-8452-74458dc6a530 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:36.948059 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e5612-ca7b-469c-8452-74458dc6a530 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:36.960198 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7e5612-ca7b-469c-8452-74458dc6a530 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:36.960802 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7e5612-ca7b-469c-8452-74458dc6a530 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:36.979900 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-ec7e5612-ca7b-469c-8452-74458dc6a530 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:09:36.979900 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:09:36.979900 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:09:36.979900 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:09:36.979900 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:09:36.979900 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:09:36.979900 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:09:36.979900 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:09:36.979900 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:09:36.979900 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:09:36.979900 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:09:36.979900 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:09:36.979900 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:09:36.979900 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:09:36.979900 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:09:36.981010 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:09:36.981010 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:09:36.981010 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:09:36.981010 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:09:36.981010 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:09:36.981010 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:09:36.981010 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:09:36.981010 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:09:36.981010 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:09:36.981010 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:09:36.981010 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:09:36.981010 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:09:36.981010 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:09:36.981010 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:09:36.981010 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:09:36.981010 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:09:36.981010 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:09:36.982294 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:09:36.982294 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:09:36.982294 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:09:36.982294 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:09:36.982294 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 614/1242] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:09:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:09:36.987144 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c5f0cd-d0a4-447c-95d1-bbb1c2bb15e2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:37.089845 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:37.097197 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c5f0cd-d0a4-447c-95d1-bbb1c2bb15e2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:37.148991 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ddae2d-5942-4e55-a78a-d57ab0ecd0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:37.150051 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ddae2d-5942-4e55-a78a-d57ab0ecd0e5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:37.150537 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ddae2d-5942-4e55-a78a-d57ab0ecd0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:37.151007 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ddae2d-5942-4e55-a78a-d57ab0ecd0e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:37.152332 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ddae2d-5942-4e55-a78a-d57ab0ecd0e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:09:37.152800 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ddae2d-5942-4e55-a78a-d57ab0ecd0e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:37.154918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ddae2d-5942-4e55-a78a-d57ab0ecd0e5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:37.170210 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 629/1243] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:09:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:09:37.175563 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7e5612-ca7b-469c-8452-74458dc6a530 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:37.258987 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:37.282245 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7e5612-ca7b-469c-8452-74458dc6a530 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:37.318081 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5689c91-f867-44dc-a5c3-ecca68429b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:37.319402 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5689c91-f867-44dc-a5c3-ecca68429b39 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:37.319704 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5689c91-f867-44dc-a5c3-ecca68429b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:37.319972 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5689c91-f867-44dc-a5c3-ecca68429b39 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:37.326839 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5689c91-f867-44dc-a5c3-ecca68429b39 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:37.328296 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5689c91-f867-44dc-a5c3-ecca68429b39 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:37.343960 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-f5689c91-f867-44dc-a5c3-ecca68429b39 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:09:37.343960 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:09:37.343960 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:09:37.343960 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:09:37.343960 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:09:37.343960 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:09:37.343960 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:09:37.343960 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:09:37.343960 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:09:37.343960 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:09:37.343960 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:09:37.343960 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:09:37.343960 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:09:37.343960 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:09:37.343960 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:09:37.345733 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:09:37.345733 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:09:37.345733 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:09:37.345733 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:09:37.345733 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:09:37.345733 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:09:37.345733 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:09:37.345733 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:09:37.345733 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:09:37.345733 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:09:37.345733 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:09:37.345733 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:09:37.345733 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:09:37.345733 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:09:37.345733 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:09:37.345733 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:09:37.345733 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:09:37.347302 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:09:37.347302 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:09:37.347302 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:09:37.347302 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:09:37.347302 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:09:37.347302 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:09:37.347302 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:09:37.347302 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:09:37.347302 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 615/1244] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 4 09:09:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:09:37.351857 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ddae2d-5942-4e55-a78a-d57ab0ecd0e5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:37.468446 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ddae2d-5942-4e55-a78a-d57ab0ecd0e5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:37.504679 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8337c589-2917-4db9-a992-1d8aec2d2e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:37.505520 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8337c589-2917-4db9-a992-1d8aec2d2e44 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:37.505801 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8337c589-2917-4db9-a992-1d8aec2d2e44 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:37.506071 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8337c589-2917-4db9-a992-1d8aec2d2e44 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:37.506890 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8337c589-2917-4db9-a992-1d8aec2d2e44 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:09:37.507190 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8337c589-2917-4db9-a992-1d8aec2d2e44 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:37.508602 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8337c589-2917-4db9-a992-1d8aec2d2e44 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:37.521014 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 630/1245] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 4 09:09:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:09:37.528782 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5689c91-f867-44dc-a5c3-ecca68429b39 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:37.641587 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5689c91-f867-44dc-a5c3-ecca68429b39 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:37.678598 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e84239-6d58-44bb-9d26-19bc7becc5be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:37.679291 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e84239-6d58-44bb-9d26-19bc7becc5be None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:37.679549 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e84239-6d58-44bb-9d26-19bc7becc5be None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:37.679828 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e84239-6d58-44bb-9d26-19bc7becc5be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:37.680659 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e84239-6d58-44bb-9d26-19bc7becc5be None admin] RBAC: Adding query filter params (role.id=5752a9bdfe4b482ab37836eba7bd536b, scope.project.id=4076e316243946e980ce47386b18b28f, user.id=293c695f15c8437d81a619f840318f44) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:09:37.680985 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e84239-6d58-44bb-9d26-19bc7becc5be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:37.682454 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e84239-6d58-44bb-9d26-19bc7becc5be None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:37.688992 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 616/1246] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 4 09:09:37 2022] GET /identity/v3/role_assignments?user.id=293c695f15c8437d81a619f840318f44&scope.project.id=4076e316243946e980ce47386b18b28f&role.id=5752a9bdfe4b482ab37836eba7bd536b => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:09:40.052973 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec2073-b323-44d8-984b-449badda8918 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:40.053884 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec2073-b323-44d8-984b-449badda8918 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:40.053884 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec2073-b323-44d8-984b-449badda8918 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:40.055167 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 631/1247] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:09:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:09:40.063337 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e93cfd9-8671-417c-a1eb-20ba1b2a7be7 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:40.063833 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e93cfd9-8671-417c-a1eb-20ba1b2a7be7 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:40.063939 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e93cfd9-8671-417c-a1eb-20ba1b2a7be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:40.069281 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:40.091730 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-2e93cfd9-8671-417c-a1eb-20ba1b2a7be7 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:09:40.092969 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-2e93cfd9-8671-417c-a1eb-20ba1b2a7be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:09:40.189789 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 617/1248] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:09:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:09:40.214988 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fec2073-b323-44d8-984b-449badda8918 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:40.235775 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:40.333662 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fec2073-b323-44d8-984b-449badda8918 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:40.373974 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62072155-a5d7-4803-915d-f7e5068488f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:40.374771 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62072155-a5d7-4803-915d-f7e5068488f9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:40.375041 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62072155-a5d7-4803-915d-f7e5068488f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:40.375317 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62072155-a5d7-4803-915d-f7e5068488f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:40.386780 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62072155-a5d7-4803-915d-f7e5068488f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:09:40.394609 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-62072155-a5d7-4803-915d-f7e5068488f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:09:40.394609 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:09:40.394609 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:09:40.394609 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:09:40.394609 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:09:40.394609 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:09:40.394609 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:09:40.394609 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:09:40.394609 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:09:40.394609 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:09:40.394609 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:09:40.394609 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:09:40.394609 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:09:40.394609 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:09:40.394609 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:09:40.394609 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:09:40.395592 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:09:40.395592 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:09:40.395592 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:09:40.395592 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:09:40.395592 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:09:40.395592 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:09:40.395592 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:09:40.395592 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:09:40.395592 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:09:40.395592 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:09:40.395592 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:09:40.395592 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:09:40.395592 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:09:40.395592 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:09:40.395592 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:09:40.395592 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:09:40.396491 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:09:40.396491 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:09:40.396491 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:09:40.396491 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:09:40.396491 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:09:40.396491 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:09:40.396491 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:09:40.396491 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:09:40.396491 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:09:40.396491 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:09:40.396491 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:09:40.396491 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:09:40.396491 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 632/1249] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 4 09:09:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:09:40.403156 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e93cfd9-8671-417c-a1eb-20ba1b2a7be7 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:40.513637 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e93cfd9-8671-417c-a1eb-20ba1b2a7be7 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:40.545091 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:40.550608 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07957b07-362c-4b19-b86f-ee35b522f6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:40.551280 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07957b07-362c-4b19-b86f-ee35b522f6ba None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:40.551567 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07957b07-362c-4b19-b86f-ee35b522f6ba None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:40.551846 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07957b07-362c-4b19-b86f-ee35b522f6ba None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:40.552728 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07957b07-362c-4b19-b86f-ee35b522f6ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:09:40.552933 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07957b07-362c-4b19-b86f-ee35b522f6ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:40.554415 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07957b07-362c-4b19-b86f-ee35b522f6ba None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:40.563447 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 618/1250] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:09:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:09:40.569501 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62072155-a5d7-4803-915d-f7e5068488f9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:40.664809 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:40.689574 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62072155-a5d7-4803-915d-f7e5068488f9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:40.724782 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79b4641-c2c8-468f-ab14-af7e9984e000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:40.725649 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79b4641-c2c8-468f-ab14-af7e9984e000 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:40.725935 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79b4641-c2c8-468f-ab14-af7e9984e000 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:40.726217 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79b4641-c2c8-468f-ab14-af7e9984e000 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:40.737851 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79b4641-c2c8-468f-ab14-af7e9984e000 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:40.739767 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79b4641-c2c8-468f-ab14-af7e9984e000 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:40.760982 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-c79b4641-c2c8-468f-ab14-af7e9984e000 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:09:40.760982 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:09:40.760982 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:09:40.760982 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:09:40.760982 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:09:40.760982 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:09:40.760982 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:09:40.760982 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:09:40.760982 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:09:40.760982 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:09:40.760982 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:09:40.760982 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:09:40.760982 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:09:40.760982 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:09:40.760982 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:09:40.770533 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:09:40.770533 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:09:40.770533 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:09:40.770533 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:09:40.770533 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:09:40.770533 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:09:40.770533 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:09:40.770533 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:09:40.770533 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:09:40.770533 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:09:40.770533 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:09:40.770533 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:09:40.770533 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:09:40.770533 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:09:40.770533 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:09:40.770533 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:09:40.770533 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:09:40.773186 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:09:40.773186 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:09:40.773186 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:09:40.773186 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:09:40.773186 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 633/1251] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:09:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:09:40.773867 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07957b07-362c-4b19-b86f-ee35b522f6ba None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:40.890701 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07957b07-362c-4b19-b86f-ee35b522f6ba None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:40.930943 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08c32b1-d623-445f-9345-59f96f8df046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:40.931540 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c32b1-d623-445f-9345-59f96f8df046 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:40.931874 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c32b1-d623-445f-9345-59f96f8df046 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:40.932179 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c32b1-d623-445f-9345-59f96f8df046 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:40.933006 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08c32b1-d623-445f-9345-59f96f8df046 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:09:40.933352 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08c32b1-d623-445f-9345-59f96f8df046 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:40.934858 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08c32b1-d623-445f-9345-59f96f8df046 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:40.950150 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 619/1252] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:09:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:09:40.957513 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79b4641-c2c8-468f-ab14-af7e9984e000 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:41.067322 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79b4641-c2c8-468f-ab14-af7e9984e000 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:41.101958 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:41.113561 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2f5496-3e08-4ec5-af31-025add3c14c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:41.114400 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f5496-3e08-4ec5-af31-025add3c14c2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:41.114684 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f5496-3e08-4ec5-af31-025add3c14c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:41.114959 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f5496-3e08-4ec5-af31-025add3c14c2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:41.127118 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2f5496-3e08-4ec5-af31-025add3c14c2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:41.129206 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2f5496-3e08-4ec5-af31-025add3c14c2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:41.151512 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-af2f5496-3e08-4ec5-af31-025add3c14c2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 04 09:09:41.151512 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:09:41.151512 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:09:41.151512 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:09:41.151512 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:09:41.151512 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:09:41.151512 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:09:41.151512 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:09:41.151512 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:09:41.151512 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:09:41.151512 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:09:41.151512 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:09:41.151512 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:09:41.151512 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:09:41.151512 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:09:41.151512 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:09:41.153075 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:09:41.153075 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:09:41.153075 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:09:41.153075 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:09:41.153075 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:09:41.153075 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:09:41.153075 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:09:41.153075 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:09:41.153075 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:09:41.153075 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:09:41.153075 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:09:41.153075 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:09:41.153075 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:09:41.153075 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:09:41.153075 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:09:41.153075 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:09:41.153075 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:09:41.153075 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:09:41.154419 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:09:41.154419 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:09:41.154419 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:09:41.154419 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:09:41.154419 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:09:41.154419 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:09:41.154419 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:09:41.154419 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:09:41.154419 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 04 09:09:41.154419 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:09:41.154419 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 634/1253] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:09:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:09:41.160079 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08c32b1-d623-445f-9345-59f96f8df046 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:41.232438 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:41.280018 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08c32b1-d623-445f-9345-59f96f8df046 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:41.316919 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d680a084-0ae9-4e2c-a10b-ab1d2a1991e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:41.317854 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d680a084-0ae9-4e2c-a10b-ab1d2a1991e1 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:41.318223 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d680a084-0ae9-4e2c-a10b-ab1d2a1991e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:41.318800 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d680a084-0ae9-4e2c-a10b-ab1d2a1991e1 None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:41.320260 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d680a084-0ae9-4e2c-a10b-ab1d2a1991e1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:09:41.320695 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d680a084-0ae9-4e2c-a10b-ab1d2a1991e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:41.322369 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d680a084-0ae9-4e2c-a10b-ab1d2a1991e1 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:41.343955 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 620/1254] 10.222.0.71 () {56 vars in 1143 bytes} [Mon Jul 4 09:09:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:09:41.351546 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2f5496-3e08-4ec5-af31-025add3c14c2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:41.466739 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2f5496-3e08-4ec5-af31-025add3c14c2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:41.501126 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20577d59-d2bc-4c8c-a9ad-8be19b02dfd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:41.501721 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20577d59-d2bc-4c8c-a9ad-8be19b02dfd2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:41.502004 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20577d59-d2bc-4c8c-a9ad-8be19b02dfd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:41.502280 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20577d59-d2bc-4c8c-a9ad-8be19b02dfd2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:41.558513 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:41.572451 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20577d59-d2bc-4c8c-a9ad-8be19b02dfd2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:41.573987 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20577d59-d2bc-4c8c-a9ad-8be19b02dfd2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:41.685076 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 635/1255] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:09:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:09:41.689365 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d680a084-0ae9-4e2c-a10b-ab1d2a1991e1 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:41.805217 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d680a084-0ae9-4e2c-a10b-ab1d2a1991e1 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:41.822327 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:41.846171 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f22cc1-09d3-4367-bf08-e9a5bfcc48f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:41.847004 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f22cc1-09d3-4367-bf08-e9a5bfcc48f5 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:41.847365 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f22cc1-09d3-4367-bf08-e9a5bfcc48f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:41.847726 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f22cc1-09d3-4367-bf08-e9a5bfcc48f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:41.859267 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f22cc1-09d3-4367-bf08-e9a5bfcc48f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:41.860877 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f22cc1-09d3-4367-bf08-e9a5bfcc48f5 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:41.881820 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-99f22cc1-09d3-4367-bf08-e9a5bfcc48f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:09:41.881820 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:09:41.881820 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:09:41.881820 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:09:41.881820 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:09:41.881820 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:09:41.881820 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:09:41.881820 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:09:41.881820 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:09:41.881820 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:09:41.881820 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:09:41.881820 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:09:41.881820 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:09:41.881820 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:09:41.881820 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:09:41.883209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:09:41.883209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:09:41.883209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:09:41.883209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:09:41.883209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:09:41.883209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:09:41.883209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:09:41.883209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:09:41.883209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:09:41.883209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:09:41.883209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:09:41.883209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:09:41.883209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:09:41.883209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:09:41.883209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:09:41.883209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:09:41.883209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:09:41.884569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:09:41.884569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:09:41.884569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:09:41.884569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:09:41.888400 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 621/1256] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:09:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:09:41.889327 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20577d59-d2bc-4c8c-a9ad-8be19b02dfd2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:41.979969 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:41.994191 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20577d59-d2bc-4c8c-a9ad-8be19b02dfd2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:42.031298 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6eee7e-81f8-46ef-84c9-824b052243b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:42.031968 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6eee7e-81f8-46ef-84c9-824b052243b1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:42.032351 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6eee7e-81f8-46ef-84c9-824b052243b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:42.032548 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6eee7e-81f8-46ef-84c9-824b052243b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:42.033291 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6eee7e-81f8-46ef-84c9-824b052243b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:09:42.033639 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6eee7e-81f8-46ef-84c9-824b052243b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:42.035041 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6eee7e-81f8-46ef-84c9-824b052243b1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:42.047892 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 636/1257] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:09:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:09:42.055032 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f22cc1-09d3-4367-bf08-e9a5bfcc48f5 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:42.172446 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f22cc1-09d3-4367-bf08-e9a5bfcc48f5 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:42.209574 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73de27bb-819d-4465-9b3c-580ba1c0edd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:42.211049 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de27bb-819d-4465-9b3c-580ba1c0edd4 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:42.211332 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de27bb-819d-4465-9b3c-580ba1c0edd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:42.211605 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de27bb-819d-4465-9b3c-580ba1c0edd4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:42.219507 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73de27bb-819d-4465-9b3c-580ba1c0edd4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:42.220979 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73de27bb-819d-4465-9b3c-580ba1c0edd4 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:42.236411 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-73de27bb-819d-4465-9b3c-580ba1c0edd4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:09:42.236411 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:09:42.236411 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:09:42.236411 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:09:42.236411 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:09:42.236411 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:09:42.236411 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:09:42.236411 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:09:42.236411 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:09:42.236411 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:09:42.236411 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:09:42.236411 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:09:42.236411 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:09:42.236411 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:09:42.236411 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:09:42.237626 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:09:42.237626 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:09:42.237626 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:09:42.237626 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:09:42.237626 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:09:42.237626 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:09:42.237626 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:09:42.237626 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:09:42.237626 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:09:42.237626 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:09:42.237626 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:09:42.237626 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:09:42.237626 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:09:42.237626 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:09:42.237626 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:09:42.237626 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:09:42.237626 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:09:42.238555 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:09:42.238555 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:09:42.238555 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:09:42.238555 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:09:42.238555 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:09:42.238555 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:09:42.238555 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:09:42.238555 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:09:42.238555 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 622/1258] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 4 09:09:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:09:42.244659 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6eee7e-81f8-46ef-84c9-824b052243b1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:42.354108 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6eee7e-81f8-46ef-84c9-824b052243b1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:42.392232 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcbd021-e2c0-4e75-81c0-dc80fbd2111d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:42.392911 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcbd021-e2c0-4e75-81c0-dc80fbd2111d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:42.393685 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcbd021-e2c0-4e75-81c0-dc80fbd2111d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:42.393685 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcbd021-e2c0-4e75-81c0-dc80fbd2111d None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:42.394298 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcbd021-e2c0-4e75-81c0-dc80fbd2111d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:09:42.394606 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcbd021-e2c0-4e75-81c0-dc80fbd2111d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:42.396047 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcbd021-e2c0-4e75-81c0-dc80fbd2111d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:42.409865 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 637/1259] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 4 09:09:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:09:42.416745 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73de27bb-819d-4465-9b3c-580ba1c0edd4 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:42.444916 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:42.521212 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73de27bb-819d-4465-9b3c-580ba1c0edd4 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:42.555617 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92058c22-aa27-4025-bd76-df154bdac84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:42.556277 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92058c22-aa27-4025-bd76-df154bdac84e None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:42.556562 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92058c22-aa27-4025-bd76-df154bdac84e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:42.556841 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92058c22-aa27-4025-bd76-df154bdac84e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:42.557704 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92058c22-aa27-4025-bd76-df154bdac84e None admin] RBAC: Adding query filter params (role.id=f36e053b442540f08041ac7878760f65, scope.project.id=4076e316243946e980ce47386b18b28f, user.id=293c695f15c8437d81a619f840318f44) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:09:42.558020 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92058c22-aa27-4025-bd76-df154bdac84e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:42.559458 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92058c22-aa27-4025-bd76-df154bdac84e None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:42.565008 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 623/1260] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 4 09:09:42 2022] GET /identity/v3/role_assignments?user.id=293c695f15c8437d81a619f840318f44&scope.project.id=4076e316243946e980ce47386b18b28f&role.id=f36e053b442540f08041ac7878760f65 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:09:44.600719 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a0d53-6ee9-4dfc-a0ab-e1805f4cc616 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:44.601375 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a0d53-6ee9-4dfc-a0ab-e1805f4cc616 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:44.601658 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a0d53-6ee9-4dfc-a0ab-e1805f4cc616 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:44.603024 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 638/1261] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:09:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:09:44.608210 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7bfecd-7719-45dc-890d-9dce09de2d2d None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:44.608646 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7bfecd-7719-45dc-890d-9dce09de2d2d None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:44.608875 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7bfecd-7719-45dc-890d-9dce09de2d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:44.625955 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-4a7bfecd-7719-45dc-890d-9dce09de2d2d None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:09:44.627142 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4a7bfecd-7719-45dc-890d-9dce09de2d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:09:44.691062 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 624/1262] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:09:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:09:44.713642 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3a0d53-6ee9-4dfc-a0ab-e1805f4cc616 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:44.724629 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:44.814632 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3a0d53-6ee9-4dfc-a0ab-e1805f4cc616 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:44.847981 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcfad80-c2a7-4e90-a261-3388c56a990c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:44.848762 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcfad80-c2a7-4e90-a261-3388c56a990c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:44.849115 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcfad80-c2a7-4e90-a261-3388c56a990c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:44.849355 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcfad80-c2a7-4e90-a261-3388c56a990c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:44.859769 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcfad80-c2a7-4e90-a261-3388c56a990c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:09:44.867471 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-9bcfad80-c2a7-4e90-a261-3388c56a990c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:09:44.867471 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:09:44.867471 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:09:44.867471 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:09:44.867471 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:09:44.867471 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:09:44.867471 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:09:44.867471 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:09:44.867471 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:09:44.867471 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:09:44.867471 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:09:44.867471 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:09:44.867471 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:09:44.867471 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:09:44.867471 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:09:44.867471 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:09:44.868598 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:09:44.868598 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:09:44.868598 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:09:44.868598 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:09:44.868598 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:09:44.868598 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:09:44.868598 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:09:44.868598 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:09:44.868598 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:09:44.868598 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:09:44.868598 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:09:44.868598 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:09:44.868598 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:09:44.868598 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:09:44.868598 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:09:44.868598 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:09:44.869697 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:09:44.869697 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:09:44.869697 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:09:44.869697 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:09:44.869697 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:09:44.869697 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:09:44.869697 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:09:44.869697 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:09:44.869697 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:09:44.869697 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:09:44.869697 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:09:44.869697 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:09:44.869697 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 639/1263] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 4 09:09:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:09:44.874744 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7bfecd-7719-45dc-890d-9dce09de2d2d None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:44.892534 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:44.971826 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7bfecd-7719-45dc-890d-9dce09de2d2d None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:45.004835 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340b7fbf-ce2d-4d3e-8a76-9d6c5d888d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:45.005522 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b7fbf-ce2d-4d3e-8a76-9d6c5d888d52 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:45.005812 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b7fbf-ce2d-4d3e-8a76-9d6c5d888d52 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:45.006114 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b7fbf-ce2d-4d3e-8a76-9d6c5d888d52 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:45.006854 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340b7fbf-ce2d-4d3e-8a76-9d6c5d888d52 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:09:45.007160 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340b7fbf-ce2d-4d3e-8a76-9d6c5d888d52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:45.008532 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340b7fbf-ce2d-4d3e-8a76-9d6c5d888d52 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:45.016413 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 625/1264] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:09:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:09:45.023173 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcfad80-c2a7-4e90-a261-3388c56a990c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:45.091733 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:45.137362 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcfad80-c2a7-4e90-a261-3388c56a990c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:45.177662 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f320cc7a-8fd2-43d1-ab7b-36fb70dabed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:45.178531 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f320cc7a-8fd2-43d1-ab7b-36fb70dabed5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:45.178817 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f320cc7a-8fd2-43d1-ab7b-36fb70dabed5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:45.179095 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f320cc7a-8fd2-43d1-ab7b-36fb70dabed5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:45.189410 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f320cc7a-8fd2-43d1-ab7b-36fb70dabed5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:45.190846 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f320cc7a-8fd2-43d1-ab7b-36fb70dabed5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:45.215652 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-f320cc7a-8fd2-43d1-ab7b-36fb70dabed5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:09:45.215652 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:09:45.215652 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:09:45.215652 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:09:45.215652 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:09:45.215652 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:09:45.215652 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:09:45.215652 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:09:45.215652 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:09:45.215652 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:09:45.215652 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:09:45.215652 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:09:45.215652 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:09:45.215652 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:09:45.215652 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:09:45.216738 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:09:45.216738 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:09:45.216738 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:09:45.216738 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:09:45.216738 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:09:45.216738 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:09:45.216738 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:09:45.216738 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:09:45.216738 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:09:45.216738 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:09:45.216738 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:09:45.216738 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:09:45.216738 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:09:45.216738 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:09:45.216738 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:09:45.216738 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:09:45.216738 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:09:45.217871 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:09:45.217871 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:09:45.217871 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:09:45.217871 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:09:45.217871 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 640/1265] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:09:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:09:45.222724 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340b7fbf-ce2d-4d3e-8a76-9d6c5d888d52 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:45.302404 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:45.322261 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340b7fbf-ce2d-4d3e-8a76-9d6c5d888d52 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:45.355365 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc311f9-db3b-48c5-8e0a-cef4e6ee142d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:45.356019 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc311f9-db3b-48c5-8e0a-cef4e6ee142d None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:45.356307 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc311f9-db3b-48c5-8e0a-cef4e6ee142d None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:45.356587 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc311f9-db3b-48c5-8e0a-cef4e6ee142d None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:45.357337 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc311f9-db3b-48c5-8e0a-cef4e6ee142d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:09:45.357649 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc311f9-db3b-48c5-8e0a-cef4e6ee142d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:45.359020 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc311f9-db3b-48c5-8e0a-cef4e6ee142d None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:45.370415 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 626/1266] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:09:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:09:45.376035 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f320cc7a-8fd2-43d1-ab7b-36fb70dabed5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:45.475483 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f320cc7a-8fd2-43d1-ab7b-36fb70dabed5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:45.494996 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:45.509649 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266b9aac-7b92-49ec-919a-1fb5530bd6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:45.510446 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266b9aac-7b92-49ec-919a-1fb5530bd6ee None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:45.510736 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266b9aac-7b92-49ec-919a-1fb5530bd6ee None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:45.511014 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266b9aac-7b92-49ec-919a-1fb5530bd6ee None admin] PATH_INFO: `/v3/users/placement` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:45.518303 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266b9aac-7b92-49ec-919a-1fb5530bd6ee None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:45.519737 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266b9aac-7b92-49ec-919a-1fb5530bd6ee None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:45.533951 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-266b9aac-7b92-49ec-919a-1fb5530bd6ee None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 04 09:09:45.533951 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:09:45.533951 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:09:45.533951 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:09:45.533951 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:09:45.533951 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:09:45.533951 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:09:45.533951 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:09:45.533951 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:09:45.533951 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:09:45.533951 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:09:45.533951 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:09:45.533951 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:09:45.533951 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:09:45.533951 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:09:45.533951 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:09:45.535116 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:09:45.535116 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:09:45.535116 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:09:45.535116 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:09:45.535116 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:09:45.535116 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:09:45.535116 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:09:45.535116 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:09:45.535116 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:09:45.535116 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:09:45.535116 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:09:45.535116 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:09:45.535116 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:09:45.535116 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:09:45.535116 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:09:45.535116 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:09:45.535116 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:09:45.535116 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:09:45.536308 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:09:45.536308 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:09:45.536308 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:09:45.536308 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:09:45.536308 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:09:45.536308 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:09:45.536308 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:09:45.536308 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:09:45.536308 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 04 09:09:45.536308 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:09:45.536308 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 641/1267] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:09:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:09:45.541131 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc311f9-db3b-48c5-8e0a-cef4e6ee142d None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:45.638005 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc311f9-db3b-48c5-8e0a-cef4e6ee142d None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:45.672154 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d605c-69a9-4e1d-ac9a-de2a523e5ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:45.672819 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d605c-69a9-4e1d-ac9a-de2a523e5ad2 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:45.673134 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d605c-69a9-4e1d-ac9a-de2a523e5ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:45.673428 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d605c-69a9-4e1d-ac9a-de2a523e5ad2 None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:45.674247 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04d605c-69a9-4e1d-ac9a-de2a523e5ad2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:09:45.674560 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04d605c-69a9-4e1d-ac9a-de2a523e5ad2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:45.675988 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04d605c-69a9-4e1d-ac9a-de2a523e5ad2 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:45.684954 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:45.694296 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 627/1268] 10.222.0.71 () {56 vars in 1143 bytes} [Mon Jul 4 09:09:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:09:45.699978 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266b9aac-7b92-49ec-919a-1fb5530bd6ee None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:45.798010 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266b9aac-7b92-49ec-919a-1fb5530bd6ee None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:45.831416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2834563-0e7e-4f60-9d50-3206a202c28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:45.832183 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2834563-0e7e-4f60-9d50-3206a202c28f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:45.832463 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2834563-0e7e-4f60-9d50-3206a202c28f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:45.832737 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2834563-0e7e-4f60-9d50-3206a202c28f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:45.843658 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2834563-0e7e-4f60-9d50-3206a202c28f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:45.845099 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2834563-0e7e-4f60-9d50-3206a202c28f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:45.861895 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-a2834563-0e7e-4f60-9d50-3206a202c28f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:09:45.861895 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:09:45.861895 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:09:45.861895 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:09:45.861895 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:09:45.861895 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:09:45.861895 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:09:45.861895 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:09:45.861895 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:09:45.861895 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:09:45.861895 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:09:45.861895 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:09:45.861895 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:09:45.861895 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:09:45.861895 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:09:45.862933 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:09:45.862933 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:09:45.862933 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:09:45.862933 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:09:45.862933 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:09:45.862933 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:09:45.862933 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:09:45.862933 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:09:45.862933 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:09:45.862933 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:09:45.862933 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:09:45.862933 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:09:45.862933 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:09:45.862933 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:09:45.862933 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:09:45.862933 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:09:45.862933 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:09:45.864055 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:09:45.864055 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:09:45.864055 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:09:45.864055 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:09:45.864055 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 642/1269] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:09:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:09:45.869770 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d605c-69a9-4e1d-ac9a-de2a523e5ad2 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:45.967391 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d605c-69a9-4e1d-ac9a-de2a523e5ad2 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:46.000967 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcb29f8-83b6-468b-a099-0edb9c961491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:46.001648 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcb29f8-83b6-468b-a099-0edb9c961491 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:46.001935 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcb29f8-83b6-468b-a099-0edb9c961491 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:46.002213 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcb29f8-83b6-468b-a099-0edb9c961491 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:46.002933 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcb29f8-83b6-468b-a099-0edb9c961491 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:09:46.003243 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcb29f8-83b6-468b-a099-0edb9c961491 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:46.004617 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcb29f8-83b6-468b-a099-0edb9c961491 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:46.015637 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 628/1270] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:09:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:09:46.023558 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2834563-0e7e-4f60-9d50-3206a202c28f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:46.055433 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:46.129972 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2834563-0e7e-4f60-9d50-3206a202c28f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:46.164232 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99808ca-1170-4ffc-9c9d-c8b4b0de56a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:46.165015 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99808ca-1170-4ffc-9c9d-c8b4b0de56a0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:46.165328 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99808ca-1170-4ffc-9c9d-c8b4b0de56a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:46.165606 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99808ca-1170-4ffc-9c9d-c8b4b0de56a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:46.171991 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99808ca-1170-4ffc-9c9d-c8b4b0de56a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:46.173448 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99808ca-1170-4ffc-9c9d-c8b4b0de56a0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:46.188043 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-a99808ca-1170-4ffc-9c9d-c8b4b0de56a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:09:46.188043 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:09:46.188043 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:09:46.188043 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:09:46.188043 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:09:46.188043 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:09:46.188043 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:09:46.188043 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:09:46.188043 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:09:46.188043 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:09:46.188043 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:09:46.188043 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:09:46.188043 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:09:46.188043 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:09:46.188043 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:09:46.189121 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:09:46.189121 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:09:46.189121 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:09:46.189121 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:09:46.189121 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:09:46.189121 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:09:46.189121 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:09:46.189121 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:09:46.189121 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:09:46.189121 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:09:46.189121 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:09:46.189121 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:09:46.189121 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:09:46.189121 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:09:46.189121 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:09:46.189121 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:09:46.189121 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:09:46.190372 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:09:46.190372 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:09:46.190372 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:09:46.190372 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:09:46.190372 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:09:46.190372 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:09:46.190372 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:09:46.190372 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:09:46.190372 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 643/1271] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 4 09:09:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:09:46.195494 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcb29f8-83b6-468b-a099-0edb9c961491 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:46.253011 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:46.294514 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcb29f8-83b6-468b-a099-0edb9c961491 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:46.326584 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f366821-05c3-4488-821b-52c5ad9243a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:46.327173 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f366821-05c3-4488-821b-52c5ad9243a1 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:46.327458 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f366821-05c3-4488-821b-52c5ad9243a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:46.327737 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f366821-05c3-4488-821b-52c5ad9243a1 None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:46.328519 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f366821-05c3-4488-821b-52c5ad9243a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:09:46.328830 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f366821-05c3-4488-821b-52c5ad9243a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:46.330293 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f366821-05c3-4488-821b-52c5ad9243a1 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:46.342839 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 629/1272] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 4 09:09:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:09:46.348786 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99808ca-1170-4ffc-9c9d-c8b4b0de56a0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:46.440983 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:46.450075 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99808ca-1170-4ffc-9c9d-c8b4b0de56a0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:46.483897 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2ca75b-381f-4ab9-b152-9cb69fd2e0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:46.484775 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ca75b-381f-4ab9-b152-9cb69fd2e0cd None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:46.485087 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ca75b-381f-4ab9-b152-9cb69fd2e0cd None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:46.485370 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ca75b-381f-4ab9-b152-9cb69fd2e0cd None admin] PATH_INFO: `/v3/projects/4076e316243946e980ce47386b18b28f/users/293c695f15c8437d81a619f840318f44/roles/f36e053b442540f08041ac7878760f65` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:46.487286 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2ca75b-381f-4ab9-b152-9cb69fd2e0cd None admin] RBAC: Authorizing `identity:create_grant(project_id=4076e316243946e980ce47386b18b28f, user_id=293c695f15c8437d81a619f840318f44, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:46.488708 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2ca75b-381f-4ab9-b152-9cb69fd2e0cd None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:46.501131 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 644/1273] 10.222.0.71 () {58 vars in 1439 bytes} [Mon Jul 4 09:09:46 2022] PUT /identity/v3/projects/4076e316243946e980ce47386b18b28f/users/293c695f15c8437d81a619f840318f44/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:09:48.492052 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec9344b-b8a9-4857-bf67-d6b434d171a5 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:48.492638 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec9344b-b8a9-4857-bf67-d6b434d171a5 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:48.492638 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec9344b-b8a9-4857-bf67-d6b434d171a5 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:48.493993 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 630/1274] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:09:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:09:48.499009 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a52117-add9-4d74-9418-00aa730554d7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:48.499421 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a52117-add9-4d74-9418-00aa730554d7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:48.499568 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a52117-add9-4d74-9418-00aa730554d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:48.516350 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-58a52117-add9-4d74-9418-00aa730554d7 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:09:48.517563 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-58a52117-add9-4d74-9418-00aa730554d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:09:48.583741 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 645/1275] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:09:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:09:48.607491 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec9344b-b8a9-4857-bf67-d6b434d171a5 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:48.706525 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec9344b-b8a9-4857-bf67-d6b434d171a5 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:48.723245 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:48.739834 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963c9577-e7eb-4cdf-9013-a2a7cb51ab1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:48.741191 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c9577-e7eb-4cdf-9013-a2a7cb51ab1e None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:48.741479 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c9577-e7eb-4cdf-9013-a2a7cb51ab1e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:48.741757 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c9577-e7eb-4cdf-9013-a2a7cb51ab1e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:48.751643 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963c9577-e7eb-4cdf-9013-a2a7cb51ab1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:09:48.759954 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-963c9577-e7eb-4cdf-9013-a2a7cb51ab1e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:09:48.759954 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:09:48.759954 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:09:48.759954 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:09:48.759954 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:09:48.759954 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:09:48.759954 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:09:48.759954 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:09:48.759954 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:09:48.759954 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:09:48.759954 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:09:48.759954 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:09:48.759954 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:09:48.759954 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:09:48.759954 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:09:48.759954 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:09:48.761178 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:09:48.761178 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:09:48.761178 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:09:48.761178 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:09:48.761178 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:09:48.761178 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:09:48.761178 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:09:48.761178 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:09:48.761178 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:09:48.761178 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:09:48.761178 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:09:48.761178 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:09:48.761178 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:09:48.761178 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:09:48.761178 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:09:48.761178 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:09:48.762334 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:09:48.762334 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:09:48.762334 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:09:48.762334 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:09:48.762334 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:09:48.762334 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:09:48.762334 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:09:48.762334 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:09:48.762334 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:09:48.762334 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:09:48.762334 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:09:48.762334 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:09:48.762334 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 631/1276] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 4 09:09:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:09:48.767019 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a52117-add9-4d74-9418-00aa730554d7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:48.845461 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:48.863330 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a52117-add9-4d74-9418-00aa730554d7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:48.896986 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b19a32-33c0-4928-8d7a-84b9ed248e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:48.897669 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b19a32-33c0-4928-8d7a-84b9ed248e8c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:48.897950 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b19a32-33c0-4928-8d7a-84b9ed248e8c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:48.898229 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b19a32-33c0-4928-8d7a-84b9ed248e8c None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:48.898969 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b19a32-33c0-4928-8d7a-84b9ed248e8c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:09:48.899278 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b19a32-33c0-4928-8d7a-84b9ed248e8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:48.900640 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b19a32-33c0-4928-8d7a-84b9ed248e8c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:48.908456 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 646/1277] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 4 09:09:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:09:48.914737 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963c9577-e7eb-4cdf-9013-a2a7cb51ab1e None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:49.016037 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963c9577-e7eb-4cdf-9013-a2a7cb51ab1e None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:49.049108 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98e2c68-b73d-45c7-a73e-d1843b6d30ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:49.049884 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e2c68-b73d-45c7-a73e-d1843b6d30ab None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:49.050168 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e2c68-b73d-45c7-a73e-d1843b6d30ab None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:49.050445 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e2c68-b73d-45c7-a73e-d1843b6d30ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:49.061443 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98e2c68-b73d-45c7-a73e-d1843b6d30ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:49.062875 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98e2c68-b73d-45c7-a73e-d1843b6d30ab None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:49.079495 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-b98e2c68-b73d-45c7-a73e-d1843b6d30ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:09:49.079495 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:09:49.079495 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:09:49.079495 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:09:49.079495 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:09:49.079495 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:09:49.079495 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:09:49.079495 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:09:49.079495 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:09:49.079495 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:09:49.079495 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:09:49.079495 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:09:49.079495 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:09:49.079495 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:09:49.079495 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:09:49.080631 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:09:49.080631 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:09:49.080631 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:09:49.080631 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:09:49.080631 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:09:49.080631 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:09:49.080631 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:09:49.080631 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:09:49.080631 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:09:49.080631 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:09:49.080631 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:09:49.080631 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:09:49.080631 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:09:49.080631 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:09:49.080631 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:09:49.080631 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:09:49.080631 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:09:49.081810 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:09:49.081810 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:09:49.081810 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:09:49.081810 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:09:49.081810 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 632/1278] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:09:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:09:49.086883 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b19a32-33c0-4928-8d7a-84b9ed248e8c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:49.201630 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b19a32-33c0-4928-8d7a-84b9ed248e8c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:49.234705 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ca478c-2ee4-4c81-85a2-f69c848c2fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:49.235364 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ca478c-2ee4-4c81-85a2-f69c848c2fa6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:49.235650 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ca478c-2ee4-4c81-85a2-f69c848c2fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:49.235931 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ca478c-2ee4-4c81-85a2-f69c848c2fa6 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:49.236678 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ca478c-2ee4-4c81-85a2-f69c848c2fa6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:09:49.236989 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ca478c-2ee4-4c81-85a2-f69c848c2fa6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:49.238390 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ca478c-2ee4-4c81-85a2-f69c848c2fa6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:49.250672 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 647/1279] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:09:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:09:49.256847 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98e2c68-b73d-45c7-a73e-d1843b6d30ab None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:49.289200 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:49.357158 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98e2c68-b73d-45c7-a73e-d1843b6d30ab None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:49.391342 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eda5ded-962a-40b1-99ab-93d65eeb1e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:49.392141 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda5ded-962a-40b1-99ab-93d65eeb1e99 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:49.392424 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda5ded-962a-40b1-99ab-93d65eeb1e99 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:49.392698 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda5ded-962a-40b1-99ab-93d65eeb1e99 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:49.400892 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eda5ded-962a-40b1-99ab-93d65eeb1e99 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:49.402370 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eda5ded-962a-40b1-99ab-93d65eeb1e99 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:49.417015 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-4eda5ded-962a-40b1-99ab-93d65eeb1e99 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 04 09:09:49.417015 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:09:49.417015 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:09:49.417015 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:09:49.417015 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:09:49.417015 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:09:49.417015 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:09:49.417015 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:09:49.417015 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:09:49.417015 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:09:49.417015 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:09:49.417015 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:09:49.417015 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:09:49.417015 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:09:49.417015 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:09:49.417015 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:09:49.418079 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:09:49.418079 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:09:49.418079 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:09:49.418079 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:09:49.418079 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:09:49.418079 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:09:49.418079 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:09:49.418079 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:09:49.418079 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:09:49.418079 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:09:49.418079 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:09:49.418079 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:09:49.418079 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:09:49.418079 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:09:49.418079 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:09:49.418079 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:09:49.418079 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:09:49.418079 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:09:49.419102 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:09:49.419102 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:09:49.419102 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:09:49.419102 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:09:49.419102 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:09:49.419102 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:09:49.419102 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:09:49.419102 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:09:49.419102 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 04 09:09:49.419102 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:09:49.419102 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 633/1280] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:09:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:09:49.423986 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ca478c-2ee4-4c81-85a2-f69c848c2fa6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:49.427722 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:49.522270 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ca478c-2ee4-4c81-85a2-f69c848c2fa6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:49.555664 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e941de4-24a4-4205-9e32-4e6d53e2fb33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:49.556261 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e941de4-24a4-4205-9e32-4e6d53e2fb33 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:49.556547 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e941de4-24a4-4205-9e32-4e6d53e2fb33 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:49.556829 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e941de4-24a4-4205-9e32-4e6d53e2fb33 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:49.557704 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e941de4-24a4-4205-9e32-4e6d53e2fb33 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:09:49.558016 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e941de4-24a4-4205-9e32-4e6d53e2fb33 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:49.559435 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e941de4-24a4-4205-9e32-4e6d53e2fb33 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:49.576023 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 648/1281] 10.222.0.71 () {56 vars in 1143 bytes} [Mon Jul 4 09:09:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:09:49.581761 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eda5ded-962a-40b1-99ab-93d65eeb1e99 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:49.672144 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:49.683030 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eda5ded-962a-40b1-99ab-93d65eeb1e99 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:49.715866 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c000f2f-7fe0-48e5-9fea-856e8027a229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:49.716506 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c000f2f-7fe0-48e5-9fea-856e8027a229 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:49.716790 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c000f2f-7fe0-48e5-9fea-856e8027a229 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:49.717091 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c000f2f-7fe0-48e5-9fea-856e8027a229 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:49.781885 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c000f2f-7fe0-48e5-9fea-856e8027a229 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:49.783333 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c000f2f-7fe0-48e5-9fea-856e8027a229 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:49.871360 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:49.882691 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 634/1282] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 4 09:09:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:09:49.888497 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e941de4-24a4-4205-9e32-4e6d53e2fb33 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:49.949509 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:49.987495 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e941de4-24a4-4205-9e32-4e6d53e2fb33 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:50.020858 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc66f6b-cd2a-4369-b3e5-ff33b2a24b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:50.021685 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc66f6b-cd2a-4369-b3e5-ff33b2a24b16 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:50.022006 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc66f6b-cd2a-4369-b3e5-ff33b2a24b16 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:50.022283 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc66f6b-cd2a-4369-b3e5-ff33b2a24b16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:50.031854 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc66f6b-cd2a-4369-b3e5-ff33b2a24b16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:50.033285 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc66f6b-cd2a-4369-b3e5-ff33b2a24b16 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:50.050047 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-0cc66f6b-cd2a-4369-b3e5-ff33b2a24b16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:09:50.050047 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:09:50.050047 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:09:50.050047 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:09:50.050047 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:09:50.050047 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:09:50.050047 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:09:50.050047 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:09:50.050047 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:09:50.050047 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:09:50.050047 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:09:50.050047 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:09:50.050047 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:09:50.050047 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:09:50.050047 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:09:50.051201 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:09:50.051201 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:09:50.051201 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:09:50.051201 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:09:50.051201 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:09:50.051201 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:09:50.051201 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:09:50.051201 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:09:50.051201 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:09:50.051201 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:09:50.051201 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:09:50.051201 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:09:50.051201 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:09:50.051201 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:09:50.051201 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:09:50.051201 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:09:50.051201 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:09:50.052300 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:09:50.052300 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:09:50.052300 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:09:50.052300 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:09:50.052300 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 649/1283] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:09:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:09:50.057383 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c000f2f-7fe0-48e5-9fea-856e8027a229 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:50.155641 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c000f2f-7fe0-48e5-9fea-856e8027a229 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:50.188234 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8ba728-02f2-46b8-aed1-772038ffbbf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:50.188883 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ba728-02f2-46b8-aed1-772038ffbbf8 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:50.189196 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ba728-02f2-46b8-aed1-772038ffbbf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:50.189475 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ba728-02f2-46b8-aed1-772038ffbbf8 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:50.190191 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8ba728-02f2-46b8-aed1-772038ffbbf8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:09:50.190495 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8ba728-02f2-46b8-aed1-772038ffbbf8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:50.191849 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8ba728-02f2-46b8-aed1-772038ffbbf8 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:50.203272 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 635/1284] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 4 09:09:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:09:50.208888 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc66f6b-cd2a-4369-b3e5-ff33b2a24b16 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:50.304120 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc66f6b-cd2a-4369-b3e5-ff33b2a24b16 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:50.317362 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:50.337679 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc190ea7-ca1a-475f-afd9-7cbbe7251590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:50.338449 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc190ea7-ca1a-475f-afd9-7cbbe7251590 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:50.338732 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc190ea7-ca1a-475f-afd9-7cbbe7251590 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:50.339034 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc190ea7-ca1a-475f-afd9-7cbbe7251590 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:50.345776 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc190ea7-ca1a-475f-afd9-7cbbe7251590 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:50.347197 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc190ea7-ca1a-475f-afd9-7cbbe7251590 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:50.360453 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-dc190ea7-ca1a-475f-afd9-7cbbe7251590 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:09:50.360453 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:09:50.360453 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:09:50.360453 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:09:50.360453 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:09:50.360453 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:09:50.360453 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:09:50.360453 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:09:50.360453 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:09:50.360453 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:09:50.360453 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:09:50.360453 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:09:50.360453 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:09:50.360453 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:09:50.360453 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:09:50.361569 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:09:50.361569 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:09:50.361569 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:09:50.361569 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:09:50.361569 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:09:50.361569 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:09:50.361569 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:09:50.361569 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:09:50.361569 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 04 09:09:50.361569 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:09:50.361569 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:09:50.361569 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:09:50.361569 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:09:50.361569 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:09:50.361569 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:09:50.361569 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:09:50.361569 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:09:50.362699 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:09:50.362699 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:09:50.362699 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:09:50.362699 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:09:50.362699 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:09:50.362699 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:09:50.362699 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:09:50.362699 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 04 09:09:50.362699 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 650/1285] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 4 09:09:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:09:50.367996 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8ba728-02f2-46b8-aed1-772038ffbbf8 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:50.418665 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:50.468663 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8ba728-02f2-46b8-aed1-772038ffbbf8 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:50.501712 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1f5851-7331-4035-8d58-c526bdbeee2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:50.502367 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f5851-7331-4035-8d58-c526bdbeee2b None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:50.502650 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f5851-7331-4035-8d58-c526bdbeee2b None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:50.502928 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f5851-7331-4035-8d58-c526bdbeee2b None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:50.503713 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1f5851-7331-4035-8d58-c526bdbeee2b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:09:50.504026 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1f5851-7331-4035-8d58-c526bdbeee2b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:50.505499 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1f5851-7331-4035-8d58-c526bdbeee2b None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:50.517828 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 636/1286] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 4 09:09:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:09:50.523959 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc190ea7-ca1a-475f-afd9-7cbbe7251590 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:50.623283 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc190ea7-ca1a-475f-afd9-7cbbe7251590 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:50.655210 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31313a80-dcc5-4fd4-9ce6-fc9d81b09851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:50.655849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31313a80-dcc5-4fd4-9ce6-fc9d81b09851 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:50.656133 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31313a80-dcc5-4fd4-9ce6-fc9d81b09851 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:50.656413 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31313a80-dcc5-4fd4-9ce6-fc9d81b09851 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:50.657269 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31313a80-dcc5-4fd4-9ce6-fc9d81b09851 None admin] RBAC: Adding query filter params (role.id=f36e053b442540f08041ac7878760f65, scope.project.id=4076e316243946e980ce47386b18b28f, user.id=293c695f15c8437d81a619f840318f44) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:09:50.657582 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31313a80-dcc5-4fd4-9ce6-fc9d81b09851 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:50.659000 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31313a80-dcc5-4fd4-9ce6-fc9d81b09851 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:50.665288 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 651/1287] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 4 09:09:50 2022] GET /identity/v3/role_assignments?user.id=293c695f15c8437d81a619f840318f44&scope.project.id=4076e316243946e980ce47386b18b28f&role.id=f36e053b442540f08041ac7878760f65 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:09:52.719497 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788b568-7684-4aac-9b40-ccffdfa07d59 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:52.720033 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788b568-7684-4aac-9b40-ccffdfa07d59 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:52.720033 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788b568-7684-4aac-9b40-ccffdfa07d59 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:52.721590 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 637/1288] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:09:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:09:52.729554 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349cb55d-b995-4ffb-a1f1-cd8613ef8c7e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:52.730044 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349cb55d-b995-4ffb-a1f1-cd8613ef8c7e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:52.730502 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349cb55d-b995-4ffb-a1f1-cd8613ef8c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:52.748151 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-349cb55d-b995-4ffb-a1f1-cd8613ef8c7e None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:09:52.749388 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-349cb55d-b995-4ffb-a1f1-cd8613ef8c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:09:52.784734 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:52.830283 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 652/1289] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:09:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:09:52.858560 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3788b568-7684-4aac-9b40-ccffdfa07d59 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:52.973114 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3788b568-7684-4aac-9b40-ccffdfa07d59 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:52.983994 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:53.013661 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983fb903-1390-42b2-8826-ecb050dcb307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:53.014470 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983fb903-1390-42b2-8826-ecb050dcb307 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:53.014757 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983fb903-1390-42b2-8826-ecb050dcb307 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:53.015037 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983fb903-1390-42b2-8826-ecb050dcb307 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:53.015699 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983fb903-1390-42b2-8826-ecb050dcb307 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:53.017094 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983fb903-1390-42b2-8826-ecb050dcb307 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:53.034505 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-983fb903-1390-42b2-8826-ecb050dcb307 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 04 09:09:53.034505 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:09:53.034505 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:09:53.034505 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:09:53.034505 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:09:53.034505 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:09:53.034505 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:09:53.034505 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:09:53.034505 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 04 09:09:53.034505 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:09:53.034505 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:09:53.034505 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:09:53.034505 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:09:53.034505 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:09:53.034505 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:09:53.034505 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:09:53.034505 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:09:53.035790 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:09:53.035790 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:09:53.035790 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:09:53.035790 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:09:53.035790 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:09:53.035790 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:09:53.035790 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:09:53.035790 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:09:53.035790 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:09:53.035790 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:09:53.035790 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:09:53.035790 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:09:53.035790 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:09:53.035790 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:09:53.035790 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:09:53.035790 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:09:53.037117 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:09:53.037117 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:09:53.037117 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:09:53.037117 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:09:53.037117 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:09:53.037117 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:09:53.037117 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:09:53.037117 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:09:53.037117 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:09:53.037117 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:09:53.037117 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:09:53.037117 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:09:53.037117 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 04 09:09:53.037117 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:09:53.037117 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 638/1290] 10.222.0.71 () {56 vars in 1117 bytes} [Mon Jul 4 09:09:52 2022] GET /identity/v3/services/placement => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:09:53.043587 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349cb55d-b995-4ffb-a1f1-cd8613ef8c7e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:53.171970 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349cb55d-b995-4ffb-a1f1-cd8613ef8c7e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:53.211021 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405620bc-4d27-46d1-901f-1e7b63a04272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:53.211689 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405620bc-4d27-46d1-901f-1e7b63a04272 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:53.212050 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405620bc-4d27-46d1-901f-1e7b63a04272 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:53.212327 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405620bc-4d27-46d1-901f-1e7b63a04272 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:53.213028 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405620bc-4d27-46d1-901f-1e7b63a04272 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:09:53.213359 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405620bc-4d27-46d1-901f-1e7b63a04272 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:53.215031 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405620bc-4d27-46d1-901f-1e7b63a04272 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:53.219867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 653/1291] 10.222.0.71 () {56 vars in 1116 bytes} [Mon Jul 4 09:09:53 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:09:53.226688 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983fb903-1390-42b2-8826-ecb050dcb307 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:53.335806 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983fb903-1390-42b2-8826-ecb050dcb307 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:53.369671 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4690b825-dfb2-469c-b48c-9eb2a7aa7d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:53.370334 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4690b825-dfb2-469c-b48c-9eb2a7aa7d80 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:53.370622 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4690b825-dfb2-469c-b48c-9eb2a7aa7d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:53.370904 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4690b825-dfb2-469c-b48c-9eb2a7aa7d80 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:53.371608 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4690b825-dfb2-469c-b48c-9eb2a7aa7d80 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:09:53.371919 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4690b825-dfb2-469c-b48c-9eb2a7aa7d80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:53.373312 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4690b825-dfb2-469c-b48c-9eb2a7aa7d80 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:53.377717 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 639/1292] 10.222.0.71 () {56 vars in 1116 bytes} [Mon Jul 4 09:09:53 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:09:55.382942 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8bd2c-c8e2-450f-b348-684dd5d86dcb None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:55.383505 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8bd2c-c8e2-450f-b348-684dd5d86dcb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:55.383505 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8bd2c-c8e2-450f-b348-684dd5d86dcb None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:55.384843 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 654/1293] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:09:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:09:55.390102 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3667983-f795-45ba-bac5-1910cf0b1f84 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:55.390503 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3667983-f795-45ba-bac5-1910cf0b1f84 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:55.390678 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3667983-f795-45ba-bac5-1910cf0b1f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:55.407805 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-d3667983-f795-45ba-bac5-1910cf0b1f84 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:09:55.408998 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d3667983-f795-45ba-bac5-1910cf0b1f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:09:55.438233 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:55.476284 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 640/1294] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:09:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:09:55.498818 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee8bd2c-c8e2-450f-b348-684dd5d86dcb None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:55.529516 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:55.601326 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee8bd2c-c8e2-450f-b348-684dd5d86dcb None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:55.635682 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fab736-28f9-4699-8ba8-cf69dac49c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:55.636275 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fab736-28f9-4699-8ba8-cf69dac49c34 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:55.636560 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fab736-28f9-4699-8ba8-cf69dac49c34 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:55.636837 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fab736-28f9-4699-8ba8-cf69dac49c34 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:55.637577 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fab736-28f9-4699-8ba8-cf69dac49c34 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:55.638940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fab736-28f9-4699-8ba8-cf69dac49c34 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:55.652354 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 655/1295] 10.222.0.71 () {60 vars in 1141 bytes} [Mon Jul 4 09:09:55 2022] POST /identity/v3/services => generated 246 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:09:57.709929 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c7ed34-d78d-43e9-907d-358b4712e577 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:57.710470 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c7ed34-d78d-43e9-907d-358b4712e577 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:57.710470 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c7ed34-d78d-43e9-907d-358b4712e577 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:57.711858 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 641/1296] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:09:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:09:57.717187 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400cf067-8714-4c45-a159-1a109433ffbe None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:57.717631 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400cf067-8714-4c45-a159-1a109433ffbe None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:57.717752 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400cf067-8714-4c45-a159-1a109433ffbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:57.734227 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-400cf067-8714-4c45-a159-1a109433ffbe None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:09:57.735401 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-400cf067-8714-4c45-a159-1a109433ffbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:09:57.807446 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 656/1297] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:09:57 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:09:57.832555 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c7ed34-d78d-43e9-907d-358b4712e577 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:57.932365 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c7ed34-d78d-43e9-907d-358b4712e577 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:57.965213 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c4c510-8064-44c4-8c2e-0b02bb95b7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:57.965945 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c4c510-8064-44c4-8c2e-0b02bb95b7d0 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:57.966234 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c4c510-8064-44c4-8c2e-0b02bb95b7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:57.966515 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c4c510-8064-44c4-8c2e-0b02bb95b7d0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:57.967166 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c4c510-8064-44c4-8c2e-0b02bb95b7d0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:57.968537 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c4c510-8064-44c4-8c2e-0b02bb95b7d0 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:57.980786 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-d0c4c510-8064-44c4-8c2e-0b02bb95b7d0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 04 09:09:57.980786 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:09:57.980786 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:09:57.980786 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:09:57.980786 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:09:57.980786 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:09:57.980786 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:09:57.980786 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:09:57.980786 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 04 09:09:57.980786 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:09:57.980786 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:09:57.980786 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:09:57.980786 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:09:57.980786 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:09:57.980786 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:09:57.980786 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:09:57.980786 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:09:57.982015 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:09:57.982015 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:09:57.982015 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:09:57.982015 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:09:57.982015 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:09:57.982015 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:09:57.982015 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:09:57.982015 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:09:57.982015 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:09:57.982015 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:09:57.982015 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:09:57.982015 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:09:57.982015 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:09:57.982015 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:09:57.982015 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:09:57.982015 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:09:57.983258 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:09:57.983258 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:09:57.983258 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:09:57.983258 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:09:57.983258 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:09:57.983258 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:09:57.983258 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:09:57.983258 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:09:57.983258 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:09:57.983258 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:09:57.983258 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:09:57.983258 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:09:57.983258 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 04 09:09:57.983258 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:09:57.983258 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 642/1298] 10.222.0.71 () {56 vars in 1117 bytes} [Mon Jul 4 09:09:57 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:09:57.987900 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400cf067-8714-4c45-a159-1a109433ffbe None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:58.007929 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:58.092806 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400cf067-8714-4c45-a159-1a109433ffbe None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:58.133764 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7342f2-10d9-47b4-8d22-971ca8903d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:58.134449 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7342f2-10d9-47b4-8d22-971ca8903d48 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:58.134737 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7342f2-10d9-47b4-8d22-971ca8903d48 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:58.135019 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7342f2-10d9-47b4-8d22-971ca8903d48 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:58.135735 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7342f2-10d9-47b4-8d22-971ca8903d48 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:09:58.136039 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7342f2-10d9-47b4-8d22-971ca8903d48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:58.137448 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7342f2-10d9-47b4-8d22-971ca8903d48 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:58.142793 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 657/1299] 10.222.0.71 () {56 vars in 1116 bytes} [Mon Jul 4 09:09:57 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:09:58.148815 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c4c510-8064-44c4-8c2e-0b02bb95b7d0 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:58.182988 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:58.253665 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c4c510-8064-44c4-8c2e-0b02bb95b7d0 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:58.286307 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa86cfc-50ba-47ed-95b6-97f25218232e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:58.286963 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa86cfc-50ba-47ed-95b6-97f25218232e None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:58.288078 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa86cfc-50ba-47ed-95b6-97f25218232e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:58.288340 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa86cfc-50ba-47ed-95b6-97f25218232e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:58.289163 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa86cfc-50ba-47ed-95b6-97f25218232e None admin] RBAC: Adding query filter params (interface=public, service_id=f6e3b02a1b484c78aac6c50fbe29decd, region_id=RegionOne) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:09:58.289482 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa86cfc-50ba-47ed-95b6-97f25218232e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:58.290865 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa86cfc-50ba-47ed-95b6-97f25218232e None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:58.295812 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 643/1300] 10.222.0.71 () {56 vars in 1251 bytes} [Mon Jul 4 09:09:58 2022] GET /identity/v3/endpoints?service_id=f6e3b02a1b484c78aac6c50fbe29decd&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:09:58.301460 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7342f2-10d9-47b4-8d22-971ca8903d48 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:58.380406 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:58.400343 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7342f2-10d9-47b4-8d22-971ca8903d48 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:58.433368 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97f14c4-c5f7-45fc-9fae-4287f0963cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:58.434023 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97f14c4-c5f7-45fc-9fae-4287f0963cf0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:58.434307 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97f14c4-c5f7-45fc-9fae-4287f0963cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:58.434587 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97f14c4-c5f7-45fc-9fae-4287f0963cf0 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:58.435260 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97f14c4-c5f7-45fc-9fae-4287f0963cf0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:58.436614 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97f14c4-c5f7-45fc-9fae-4287f0963cf0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:58.443252 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 658/1301] 10.222.0.71 () {56 vars in 1087 bytes} [Mon Jul 4 09:09:58 2022] GET /identity/v3/services => generated 2191 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:10:00.427233 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18443a93-af9f-4401-96c7-f094f624907d None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:10:00.427693 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18443a93-af9f-4401-96c7-f094f624907d None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:10:00.427785 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18443a93-af9f-4401-96c7-f094f624907d None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:10:00.429208 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 644/1302] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:10:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:10:00.434168 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30684700-bb82-40d3-bf0d-03f7797de404 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:10:00.434573 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30684700-bb82-40d3-bf0d-03f7797de404 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:10:00.434734 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30684700-bb82-40d3-bf0d-03f7797de404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:10:00.451656 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-30684700-bb82-40d3-bf0d-03f7797de404 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:10:00.452844 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-30684700-bb82-40d3-bf0d-03f7797de404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:10:00.519911 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 659/1303] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:10:00 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:10:00.542926 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18443a93-af9f-4401-96c7-f094f624907d None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:10:00.636070 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:10:00.643120 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18443a93-af9f-4401-96c7-f094f624907d None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:10:00.675196 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe8d7b2-6131-404a-af68-c7a250dc834a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:10:00.675994 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe8d7b2-6131-404a-af68-c7a250dc834a None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:10:00.676277 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe8d7b2-6131-404a-af68-c7a250dc834a None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:10:00.676556 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe8d7b2-6131-404a-af68-c7a250dc834a None admin] PATH_INFO: `/v3/services/placement` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:10:00.677237 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe8d7b2-6131-404a-af68-c7a250dc834a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:10:00.679097 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe8d7b2-6131-404a-af68-c7a250dc834a None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:10:00.690711 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-8fe8d7b2-6131-404a-af68-c7a250dc834a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 04 09:10:00.690711 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:10:00.690711 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:10:00.690711 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:10:00.690711 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:10:00.690711 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:10:00.690711 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:10:00.690711 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:10:00.690711 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 04 09:10:00.690711 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:10:00.690711 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:10:00.690711 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:10:00.690711 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:10:00.690711 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:10:00.690711 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:10:00.690711 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:10:00.690711 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:10:00.691895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:10:00.691895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:10:00.691895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:10:00.691895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:10:00.691895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:10:00.691895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:10:00.691895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:10:00.691895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:10:00.691895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:10:00.691895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:10:00.691895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:10:00.691895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:10:00.691895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:10:00.691895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:10:00.691895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:10:00.691895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:10:00.693154 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:10:00.693154 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 04 09:10:00.693154 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:10:00.693154 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:10:00.693154 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:10:00.693154 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:10:00.693154 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:10:00.693154 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:10:00.693154 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:10:00.693154 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:10:00.693154 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:10:00.693154 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:10:00.693154 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 04 09:10:00.693154 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 04 09:10:00.693154 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 645/1304] 10.222.0.71 () {56 vars in 1117 bytes} [Mon Jul 4 09:10:00 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:10:00.698638 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30684700-bb82-40d3-bf0d-03f7797de404 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:10:00.772307 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:10:00.794593 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30684700-bb82-40d3-bf0d-03f7797de404 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:10:00.827791 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757a6038-5c97-4e94-a08d-584b5631eb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:10:00.828425 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a6038-5c97-4e94-a08d-584b5631eb48 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:10:00.828711 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a6038-5c97-4e94-a08d-584b5631eb48 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:10:00.828991 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a6038-5c97-4e94-a08d-584b5631eb48 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:10:00.829719 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757a6038-5c97-4e94-a08d-584b5631eb48 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:10:00.830027 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757a6038-5c97-4e94-a08d-584b5631eb48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:10:00.831385 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757a6038-5c97-4e94-a08d-584b5631eb48 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:10:00.836411 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 660/1305] 10.222.0.71 () {56 vars in 1116 bytes} [Mon Jul 4 09:10:00 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:10:00.842354 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe8d7b2-6131-404a-af68-c7a250dc834a None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:10:00.940493 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe8d7b2-6131-404a-af68-c7a250dc834a None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:10:00.972851 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85d2180-d422-4e62-9388-16e0b3b0efea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:10:00.973522 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d2180-d422-4e62-9388-16e0b3b0efea None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:10:00.973812 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d2180-d422-4e62-9388-16e0b3b0efea None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:10:00.974091 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d2180-d422-4e62-9388-16e0b3b0efea None admin] PATH_INFO: `/v3/endpoints` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:10:00.975511 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85d2180-d422-4e62-9388-16e0b3b0efea None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:10:00.976877 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85d2180-d422-4e62-9388-16e0b3b0efea None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:10:00.994839 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 646/1306] 10.222.0.71 () {60 vars in 1144 bytes} [Mon Jul 4 09:10:00 2022] POST /identity/v3/endpoints => generated 331 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:10:10.833447 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=30 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:10:10.989873 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:11:03.927170 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143bd2ed-ba4b-4ed1-940f-d914e6827743 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:11:03.928751 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143bd2ed-ba4b-4ed1-940f-d914e6827743 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:11:03.929685 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143bd2ed-ba4b-4ed1-940f-d914e6827743 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:11:03.935095 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 661/1307] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:11:03 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:11:03.936559 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b59bfe-3bed-417d-81e2-5a5d75c3d339 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:11:03.936977 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b59bfe-3bed-417d-81e2-5a5d75c3d339 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:11:03.937194 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b59bfe-3bed-417d-81e2-5a5d75c3d339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:11:04.040911 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-f3b59bfe-3bed-417d-81e2-5a5d75c3d339 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:11:04.044652 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f3b59bfe-3bed-417d-81e2-5a5d75c3d339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:11:04.284536 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 647/1308] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:11:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:11:04.339669 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d317cd6f-5477-4534-8713-d21ad1935880 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:11:04.340108 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d317cd6f-5477-4534-8713-d21ad1935880 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:11:04.340620 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d317cd6f-5477-4534-8713-d21ad1935880 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:11:04.342729 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 662/1309] 10.222.0.71 () {54 vars in 964 bytes} [Mon Jul 4 09:11:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:11:04.348916 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52070505-83a7-411d-b61f-ecbcac1b7597 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:11:04.350698 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52070505-83a7-411d-b61f-ecbcac1b7597 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:11:04.351247 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52070505-83a7-411d-b61f-ecbcac1b7597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:11:04.433119 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-52070505-83a7-411d-b61f-ecbcac1b7597 None None] MFA Rules not processed for user `15818e58661646f381da0973d4c2ff64`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:11:04.434704 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-52070505-83a7-411d-b61f-ecbcac1b7597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:11:04.456285 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:11:04.527720 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d317cd6f-5477-4534-8713-d21ad1935880 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:11:04.529874 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d317cd6f-5477-4534-8713-d21ad1935880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:11:04.534093 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 648/1310] 10.222.0.71 () {58 vars in 1061 bytes} [Mon Jul 4 09:11:04 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:11:04.663991 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d317cd6f-5477-4534-8713-d21ad1935880 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:11:04.747553 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca03ac7c-e3cc-4cf8-9d9c-01e7eb0d300f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:11:04.748217 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca03ac7c-e3cc-4cf8-9d9c-01e7eb0d300f service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:11:04.748500 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca03ac7c-e3cc-4cf8-9d9c-01e7eb0d300f service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:11:04.748779 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca03ac7c-e3cc-4cf8-9d9c-01e7eb0d300f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:11:04.750491 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ca03ac7c-e3cc-4cf8-9d9c-01e7eb0d300f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:11:04.807013 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:11:04.872956 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca03ac7c-e3cc-4cf8-9d9c-01e7eb0d300f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:11:04.874438 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca03ac7c-e3cc-4cf8-9d9c-01e7eb0d300f service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:11:04.895587 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 663/1311] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:11:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:11:07.333109 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f5f0a-21c9-4ad2-b58e-dd674f953edd None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:11:07.334158 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f5f0a-21c9-4ad2-b58e-dd674f953edd None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:11:07.334266 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f5f0a-21c9-4ad2-b58e-dd674f953edd None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:11:07.335565 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 649/1312] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:11:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:11:07.340636 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e51f75-e89c-4cf3-b279-e0065e8d23d1 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:11:07.341082 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e51f75-e89c-4cf3-b279-e0065e8d23d1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:11:07.341250 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e51f75-e89c-4cf3-b279-e0065e8d23d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:11:07.428166 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b7e51f75-e89c-4cf3-b279-e0065e8d23d1 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:11:07.429414 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b7e51f75-e89c-4cf3-b279-e0065e8d23d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:11:07.526475 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 664/1313] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:11:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:11:07.564891 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5f5f0a-21c9-4ad2-b58e-dd674f953edd None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:11:07.566549 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-3c5f5f0a-21c9-4ad2-b58e-dd674f953edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:11:07.675722 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5f5f0a-21c9-4ad2-b58e-dd674f953edd None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:11:07.753287 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:11:07.756994 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd504c9e-6f26-4aa6-9343-247fc5fe08fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:11:07.757998 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd504c9e-6f26-4aa6-9343-247fc5fe08fc service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:11:07.758480 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd504c9e-6f26-4aa6-9343-247fc5fe08fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:11:07.758954 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd504c9e-6f26-4aa6-9343-247fc5fe08fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:11:07.760668 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-fd504c9e-6f26-4aa6-9343-247fc5fe08fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:11:07.878280 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd504c9e-6f26-4aa6-9343-247fc5fe08fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:11:07.879729 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd504c9e-6f26-4aa6-9343-247fc5fe08fc service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:11:07.898844 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 650/1314] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:11:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:11:10.228238 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2e2c5-43fb-4c13-af09-2bd66513d9ea None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:11:10.228775 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2e2c5-43fb-4c13-af09-2bd66513d9ea None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:11:10.228775 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2e2c5-43fb-4c13-af09-2bd66513d9ea None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:11:10.230187 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 665/1315] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:11:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:11:10.235158 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64721a3-cd0b-411f-a3ab-44f4dd74feff None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:11:10.235589 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64721a3-cd0b-411f-a3ab-44f4dd74feff None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:11:10.235749 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64721a3-cd0b-411f-a3ab-44f4dd74feff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:11:10.301029 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-b64721a3-cd0b-411f-a3ab-44f4dd74feff None None] MFA Rules not processed for user `a8b858d3091e4e51b82bacac35f8daa7`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:11:10.302945 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b64721a3-cd0b-411f-a3ab-44f4dd74feff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:11:10.355154 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:11:10.400725 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 651/1316] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:11:10 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:11:10.447687 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde2e2c5-43fb-4c13-af09-2bd66513d9ea None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:11:10.449284 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bde2e2c5-43fb-4c13-af09-2bd66513d9ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:11:10.459344 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:11:10.553816 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde2e2c5-43fb-4c13-af09-2bd66513d9ea None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:11:10.625246 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae5e6b7-37ca-44c6-b6dd-5f92ea34be30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:11:10.625823 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae5e6b7-37ca-44c6-b6dd-5f92ea34be30 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:11:10.626110 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae5e6b7-37ca-44c6-b6dd-5f92ea34be30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:11:10.626391 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae5e6b7-37ca-44c6-b6dd-5f92ea34be30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:11:10.628031 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-eae5e6b7-37ca-44c6-b6dd-5f92ea34be30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:11:10.689391 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:11:10.736511 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae5e6b7-37ca-44c6-b6dd-5f92ea34be30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:11:10.737963 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae5e6b7-37ca-44c6-b6dd-5f92ea34be30 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:11:10.756022 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 666/1317] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:11:10 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:11:13.363133 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e5bd3-73ea-4222-9a55-9bc5f749c654 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:11:13.363705 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e5bd3-73ea-4222-9a55-9bc5f749c654 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:11:13.363705 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e5bd3-73ea-4222-9a55-9bc5f749c654 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:11:13.365084 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 652/1318] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:11:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:11:13.370460 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021408f-230d-419a-876a-dc17408e5413 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:11:13.370875 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021408f-230d-419a-876a-dc17408e5413 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:11:13.371029 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021408f-230d-419a-876a-dc17408e5413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:11:13.438017 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2021408f-230d-419a-876a-dc17408e5413 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:11:13.439202 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2021408f-230d-419a-876a-dc17408e5413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:11:13.532264 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 667/1319] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:11:13 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:11:13.571116 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778e5bd3-73ea-4222-9a55-9bc5f749c654 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:11:13.572640 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-778e5bd3-73ea-4222-9a55-9bc5f749c654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:11:13.672960 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778e5bd3-73ea-4222-9a55-9bc5f749c654 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:11:13.746819 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dd14a6-4112-4ad9-b457-ceda25d2feb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:11:13.747457 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dd14a6-4112-4ad9-b457-ceda25d2feb1 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:11:13.747738 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dd14a6-4112-4ad9-b457-ceda25d2feb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:11:13.748012 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dd14a6-4112-4ad9-b457-ceda25d2feb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:11:13.749670 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-14dd14a6-4112-4ad9-b457-ceda25d2feb1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:11:13.757753 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:11:13.858262 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dd14a6-4112-4ad9-b457-ceda25d2feb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:11:13.859686 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dd14a6-4112-4ad9-b457-ceda25d2feb1 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:11:13.877363 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 653/1320] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:11:13 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:11:16.593841 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59aca48-4005-4fad-8b1e-e8c8f9dad7f9 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:11:16.594380 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59aca48-4005-4fad-8b1e-e8c8f9dad7f9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:11:16.594380 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59aca48-4005-4fad-8b1e-e8c8f9dad7f9 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:11:16.595771 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 668/1321] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:11:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:11:16.601787 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc596b3-fc2f-4d13-b7ba-da2f1d0b519c None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:11:16.602202 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc596b3-fc2f-4d13-b7ba-da2f1d0b519c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:11:16.602359 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc596b3-fc2f-4d13-b7ba-da2f1d0b519c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:11:16.666444 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-cfc596b3-fc2f-4d13-b7ba-da2f1d0b519c None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:11:16.667572 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-cfc596b3-fc2f-4d13-b7ba-da2f1d0b519c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:11:16.726265 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:11:16.765319 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 654/1322] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:11:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:11:16.802068 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59aca48-4005-4fad-8b1e-e8c8f9dad7f9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:11:16.803659 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e59aca48-4005-4fad-8b1e-e8c8f9dad7f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:11:16.825006 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:11:16.921919 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59aca48-4005-4fad-8b1e-e8c8f9dad7f9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:11:17.001663 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b141c657-262a-450d-aa71-4624d14afc65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:11:17.002318 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141c657-262a-450d-aa71-4624d14afc65 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:11:17.002600 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141c657-262a-450d-aa71-4624d14afc65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:11:17.002877 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141c657-262a-450d-aa71-4624d14afc65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:11:17.004540 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b141c657-262a-450d-aa71-4624d14afc65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:11:17.078783 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:11:17.119830 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b141c657-262a-450d-aa71-4624d14afc65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:11:17.121300 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b141c657-262a-450d-aa71-4624d14afc65 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:11:17.140069 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 669/1323] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:11:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:11:19.289472 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a20ae-1e66-4916-a847-166da0c7abcd None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:11:19.290011 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a20ae-1e66-4916-a847-166da0c7abcd None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:11:19.290011 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a20ae-1e66-4916-a847-166da0c7abcd None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:11:19.291301 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 655/1324] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:11:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:11:19.296559 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa961fbb-d1aa-4c84-a311-15fa23c9f49b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:11:19.297002 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa961fbb-d1aa-4c84-a311-15fa23c9f49b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:11:19.297128 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa961fbb-d1aa-4c84-a311-15fa23c9f49b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:11:19.361639 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-aa961fbb-d1aa-4c84-a311-15fa23c9f49b None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:11:19.362844 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-aa961fbb-d1aa-4c84-a311-15fa23c9f49b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:11:19.463557 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 670/1325] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:11:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:11:19.499420 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3a20ae-1e66-4916-a847-166da0c7abcd None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:11:19.500948 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0b3a20ae-1e66-4916-a847-166da0c7abcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:11:19.604724 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3a20ae-1e66-4916-a847-166da0c7abcd None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:11:19.677942 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf8fecb-da5e-471d-84f4-c82ee3a52ca5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:11:19.678682 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf8fecb-da5e-471d-84f4-c82ee3a52ca5 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:11:19.678968 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf8fecb-da5e-471d-84f4-c82ee3a52ca5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:11:19.679246 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf8fecb-da5e-471d-84f4-c82ee3a52ca5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:11:19.681009 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ccf8fecb-da5e-471d-84f4-c82ee3a52ca5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:11:19.700437 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:11:19.796777 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf8fecb-da5e-471d-84f4-c82ee3a52ca5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:11:19.798277 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf8fecb-da5e-471d-84f4-c82ee3a52ca5 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:11:19.817139 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 656/1326] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:11:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:11:21.924311 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4db834-56e0-42fd-81a1-bbe5bc516e4d None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:11:21.924915 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4db834-56e0-42fd-81a1-bbe5bc516e4d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:11:21.924915 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4db834-56e0-42fd-81a1-bbe5bc516e4d None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:11:21.926271 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 671/1327] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:11:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:11:21.932429 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77bbd1-ca0f-4ee2-8db6-1ef74091e4ee None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:11:21.932871 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77bbd1-ca0f-4ee2-8db6-1ef74091e4ee None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:11:21.932985 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77bbd1-ca0f-4ee2-8db6-1ef74091e4ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:11:22.012673 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-db77bbd1-ca0f-4ee2-8db6-1ef74091e4ee None None] MFA Rules not processed for user `a8b858d3091e4e51b82bacac35f8daa7`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:11:22.013837 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-db77bbd1-ca0f-4ee2-8db6-1ef74091e4ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:11:22.081889 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:11:22.117636 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 657/1328] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:11:21 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:11:22.157299 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4db834-56e0-42fd-81a1-bbe5bc516e4d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:11:22.158828 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2d4db834-56e0-42fd-81a1-bbe5bc516e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:11:22.190440 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:11:22.267142 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4db834-56e0-42fd-81a1-bbe5bc516e4d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:11:22.343389 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9e13bd-d920-4d24-94ee-4e99f223c886 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:11:22.344049 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e13bd-d920-4d24-94ee-4e99f223c886 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:11:22.344334 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e13bd-d920-4d24-94ee-4e99f223c886 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:11:22.344614 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e13bd-d920-4d24-94ee-4e99f223c886 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:11:22.346294 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fb9e13bd-d920-4d24-94ee-4e99f223c886 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:11:22.426697 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:11:22.459541 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9e13bd-d920-4d24-94ee-4e99f223c886 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:11:22.460972 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9e13bd-d920-4d24-94ee-4e99f223c886 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:11:22.484199 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 672/1329] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:11:22 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:11:24.776357 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d770d0b-92ea-4639-9881-f40990a0d140 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:11:24.776928 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d770d0b-92ea-4639-9881-f40990a0d140 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:11:24.776928 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d770d0b-92ea-4639-9881-f40990a0d140 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:11:24.778354 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 658/1330] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:11:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:11:24.783410 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7231412-cac2-4b6b-af32-90e1d308498b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:11:24.783827 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7231412-cac2-4b6b-af32-90e1d308498b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:11:24.783975 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7231412-cac2-4b6b-af32-90e1d308498b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:11:24.847736 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e7231412-cac2-4b6b-af32-90e1d308498b None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:11:24.848938 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e7231412-cac2-4b6b-af32-90e1d308498b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:11:24.944527 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 673/1331] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:11:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:11:24.981011 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d770d0b-92ea-4639-9881-f40990a0d140 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:11:24.982591 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0d770d0b-92ea-4639-9881-f40990a0d140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:11:25.085924 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d770d0b-92ea-4639-9881-f40990a0d140 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:11:25.159139 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f62a8aa-70d9-46e3-b9b3-84b0a4c252dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:11:25.159786 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f62a8aa-70d9-46e3-b9b3-84b0a4c252dd service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:11:25.160073 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f62a8aa-70d9-46e3-b9b3-84b0a4c252dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:11:25.160356 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f62a8aa-70d9-46e3-b9b3-84b0a4c252dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:11:25.162231 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-2f62a8aa-70d9-46e3-b9b3-84b0a4c252dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:11:25.188348 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:11:25.282140 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f62a8aa-70d9-46e3-b9b3-84b0a4c252dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:11:25.283590 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f62a8aa-70d9-46e3-b9b3-84b0a4c252dd service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:11:25.301513 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 659/1332] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:11:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:11:27.398777 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e99fcb-7d33-44bc-b5ce-22f3ed8b571a None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:11:27.399193 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e99fcb-7d33-44bc-b5ce-22f3ed8b571a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:11:27.399350 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e99fcb-7d33-44bc-b5ce-22f3ed8b571a None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:11:27.400735 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 674/1333] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:11:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:11:27.406003 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a520a48-e5fc-4628-a6db-248dede9cb45 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:11:27.406428 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a520a48-e5fc-4628-a6db-248dede9cb45 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:11:27.406582 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a520a48-e5fc-4628-a6db-248dede9cb45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:11:27.472177 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-7a520a48-e5fc-4628-a6db-248dede9cb45 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:11:27.473480 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7a520a48-e5fc-4628-a6db-248dede9cb45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:11:27.547205 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:11:27.570718 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 660/1334] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:11:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:11:27.625782 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd507f-0747-4e9b-86c8-fa0251571cb2 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:11:27.626075 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd507f-0747-4e9b-86c8-fa0251571cb2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:11:27.626358 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd507f-0747-4e9b-86c8-fa0251571cb2 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:11:27.627818 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 675/1335] 10.222.0.71 () {54 vars in 964 bytes} [Mon Jul 4 09:11:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:11:27.634303 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b5c49-282f-44fc-bf12-2a3f051a9f1c None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:11:27.634600 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b5c49-282f-44fc-bf12-2a3f051a9f1c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:11:27.634886 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b5c49-282f-44fc-bf12-2a3f051a9f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:11:27.710377 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-c31b5c49-282f-44fc-bf12-2a3f051a9f1c None None] MFA Rules not processed for user `15818e58661646f381da0973d4c2ff64`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:11:27.711584 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c31b5c49-282f-44fc-bf12-2a3f051a9f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:11:27.788038 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 661/1336] 10.222.0.71 () {58 vars in 1061 bytes} [Mon Jul 4 09:11:27 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:11:27.797057 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd507f-0747-4e9b-86c8-fa0251571cb2 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:11:27.798597 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-27fd507f-0747-4e9b-86c8-fa0251571cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:11:27.839613 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:11:27.903619 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd507f-0747-4e9b-86c8-fa0251571cb2 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:11:27.976985 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e284a146-25e9-449a-b4f2-a15fef40f054 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:11:27.977597 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284a146-25e9-449a-b4f2-a15fef40f054 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:11:27.977883 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284a146-25e9-449a-b4f2-a15fef40f054 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:11:27.978160 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284a146-25e9-449a-b4f2-a15fef40f054 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:11:27.979818 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e284a146-25e9-449a-b4f2-a15fef40f054 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:11:28.063023 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:11:28.089961 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e284a146-25e9-449a-b4f2-a15fef40f054 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:11:28.091404 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e284a146-25e9-449a-b4f2-a15fef40f054 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:11:28.110090 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 676/1337] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:11:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:11:30.994434 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e62b7-9a32-422a-8387-01084a7368e4 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:11:30.994993 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e62b7-9a32-422a-8387-01084a7368e4 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:11:30.994993 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e62b7-9a32-422a-8387-01084a7368e4 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:11:30.996375 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 662/1338] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:11:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:11:31.001748 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8149d480-086c-48e5-acc6-ee4186c41a90 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:11:31.002172 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8149d480-086c-48e5-acc6-ee4186c41a90 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:11:31.002320 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8149d480-086c-48e5-acc6-ee4186c41a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:11:31.067531 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8149d480-086c-48e5-acc6-ee4186c41a90 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:11:31.068759 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8149d480-086c-48e5-acc6-ee4186c41a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:11:31.174539 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 677/1339] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:11:31 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:11:31.211141 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774e62b7-9a32-422a-8387-01084a7368e4 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:11:31.212669 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-774e62b7-9a32-422a-8387-01084a7368e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:11:31.288019 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:11:31.315694 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774e62b7-9a32-422a-8387-01084a7368e4 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:11:31.388364 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a384f1f-73ab-443f-a5ca-927f2c11dd33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:11:31.389017 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a384f1f-73ab-443f-a5ca-927f2c11dd33 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:11:31.389326 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a384f1f-73ab-443f-a5ca-927f2c11dd33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:11:31.389597 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a384f1f-73ab-443f-a5ca-927f2c11dd33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:11:31.391240 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-9a384f1f-73ab-443f-a5ca-927f2c11dd33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:11:31.500367 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a384f1f-73ab-443f-a5ca-927f2c11dd33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:11:31.501834 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a384f1f-73ab-443f-a5ca-927f2c11dd33 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:11:31.519722 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 663/1340] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:11:31 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:11:35.643703 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e45bb-eed5-47ae-8325-fd9f4b2f37a9 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:11:35.644222 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e45bb-eed5-47ae-8325-fd9f4b2f37a9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:11:35.644222 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e45bb-eed5-47ae-8325-fd9f4b2f37a9 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:11:35.645621 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 678/1341] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:11:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:11:35.651595 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43545cfb-52c1-4e65-b83d-fea2089b7d65 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:11:35.652060 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43545cfb-52c1-4e65-b83d-fea2089b7d65 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:11:35.652159 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43545cfb-52c1-4e65-b83d-fea2089b7d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:11:35.658324 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:11:35.725473 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-43545cfb-52c1-4e65-b83d-fea2089b7d65 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:11:35.726695 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-43545cfb-52c1-4e65-b83d-fea2089b7d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:11:35.838944 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 664/1342] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:11:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:11:35.878136 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e45bb-eed5-47ae-8325-fd9f4b2f37a9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:11:35.880208 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9a5e45bb-eed5-47ae-8325-fd9f4b2f37a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:11:35.932397 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:11:35.978526 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e45bb-eed5-47ae-8325-fd9f4b2f37a9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:11:35.982028 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fee25b-4ade-401c-91ee-64907867a669 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:11:35.982672 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fee25b-4ade-401c-91ee-64907867a669 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:11:35.982953 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fee25b-4ade-401c-91ee-64907867a669 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:11:35.983233 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fee25b-4ade-401c-91ee-64907867a669 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:11:35.992215 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fee25b-4ade-401c-91ee-64907867a669 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:11:35.993663 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fee25b-4ade-401c-91ee-64907867a669 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:11:36.002843 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 679/1343] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:11:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:11:38.662615 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c071f149-45ec-4edd-b0dc-e8f837084404 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:11:38.663179 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c071f149-45ec-4edd-b0dc-e8f837084404 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:11:38.663179 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c071f149-45ec-4edd-b0dc-e8f837084404 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:11:38.664513 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 665/1344] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:11:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:11:38.669968 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06916b-e00f-418d-b35d-8da296d0e08d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:11:38.670396 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06916b-e00f-418d-b35d-8da296d0e08d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:11:38.670530 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06916b-e00f-418d-b35d-8da296d0e08d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:11:38.700318 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-dc06916b-e00f-418d-b35d-8da296d0e08d None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:11:38.701543 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dc06916b-e00f-418d-b35d-8da296d0e08d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:11:38.780194 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 680/1345] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:11:38 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:11:38.819290 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c071f149-45ec-4edd-b0dc-e8f837084404 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:11:38.830555 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c071f149-45ec-4edd-b0dc-e8f837084404 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:11:38.834340 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eeea4d-d3ae-4fea-ab48-6a5f95ddbb23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:11:38.834994 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eeea4d-d3ae-4fea-ab48-6a5f95ddbb23 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:11:38.835429 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eeea4d-d3ae-4fea-ab48-6a5f95ddbb23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:11:38.835726 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eeea4d-d3ae-4fea-ab48-6a5f95ddbb23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:11:38.845964 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eeea4d-d3ae-4fea-ab48-6a5f95ddbb23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:11:38.847426 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eeea4d-d3ae-4fea-ab48-6a5f95ddbb23 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:11:38.858223 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 666/1346] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:11:38 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:11:42.851253 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8002b54-adaa-4273-81fa-78f908289c75 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:11:42.851787 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8002b54-adaa-4273-81fa-78f908289c75 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:11:42.851869 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8002b54-adaa-4273-81fa-78f908289c75 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:11:42.853253 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 681/1347] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:11:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:11:42.858743 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279feeb1-4a67-48cc-974f-58e310142cc2 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:11:42.859175 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279feeb1-4a67-48cc-974f-58e310142cc2 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:11:42.859312 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279feeb1-4a67-48cc-974f-58e310142cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:11:42.877142 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-279feeb1-4a67-48cc-974f-58e310142cc2 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:11:42.878165 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-279feeb1-4a67-48cc-974f-58e310142cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:11:42.886984 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 667/1348] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:11:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:11:42.924453 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8002b54-adaa-4273-81fa-78f908289c75 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:11:42.935488 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8002b54-adaa-4273-81fa-78f908289c75 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:11:42.939022 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105a5707-62d3-4a89-bb6d-787a30544d8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:11:42.939671 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a5707-62d3-4a89-bb6d-787a30544d8c service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:11:42.939958 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a5707-62d3-4a89-bb6d-787a30544d8c service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:11:42.940242 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a5707-62d3-4a89-bb6d-787a30544d8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:11:42.950173 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105a5707-62d3-4a89-bb6d-787a30544d8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:11:42.951603 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105a5707-62d3-4a89-bb6d-787a30544d8c service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:11:42.961874 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 682/1349] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:11:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:11:47.944829 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cbf844-a9d7-4862-9647-083941fcdb9f None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:11:47.945369 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cbf844-a9d7-4862-9647-083941fcdb9f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:11:47.945467 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cbf844-a9d7-4862-9647-083941fcdb9f None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:11:47.947595 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 668/1350] 10.222.0.71 () {54 vars in 921 bytes} [Mon Jul 4 09:11:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:11:47.964453 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e537433-2728-4c85-9928-93ae0953b637 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:11:47.966785 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e537433-2728-4c85-9928-93ae0953b637 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:11:47.967506 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e537433-2728-4c85-9928-93ae0953b637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:11:48.078093 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5e537433-2728-4c85-9928-93ae0953b637 None None] MFA Rules not processed for user `293c695f15c8437d81a619f840318f44`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:11:48.079687 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5e537433-2728-4c85-9928-93ae0953b637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:11:48.117443 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:11:48.169944 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 683/1351] 10.222.0.71 () {58 vars in 1018 bytes} [Mon Jul 4 09:11:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:11:52.491318 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9022fd37-b56b-4de1-8333-040ad17f0c6e None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:11:52.491914 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9022fd37-b56b-4de1-8333-040ad17f0c6e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:11:52.491914 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9022fd37-b56b-4de1-8333-040ad17f0c6e None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:11:52.493546 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 669/1352] 10.222.0.71 () {54 vars in 921 bytes} [Mon Jul 4 09:11:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:11:52.498894 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce4623-abdd-482d-9ae4-f312989cce31 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:11:52.499856 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce4623-abdd-482d-9ae4-f312989cce31 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:11:52.500535 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce4623-abdd-482d-9ae4-f312989cce31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:11:52.523985 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-20ce4623-abdd-482d-9ae4-f312989cce31 None None] MFA Rules not processed for user `293c695f15c8437d81a619f840318f44`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:11:52.525713 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-20ce4623-abdd-482d-9ae4-f312989cce31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:11:52.536302 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 684/1353] 10.222.0.71 () {58 vars in 1018 bytes} [Mon Jul 4 09:11:52 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:11:52.872216 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=72 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:11:53.963796 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b0960a-511e-4ee3-b289-d8da5010e419 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:11:53.964505 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b0960a-511e-4ee3-b289-d8da5010e419 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:11:53.964505 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b0960a-511e-4ee3-b289-d8da5010e419 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:11:53.965767 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 670/1354] 10.222.0.71 () {54 vars in 921 bytes} [Mon Jul 4 09:11:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:11:53.971109 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6185d149-d550-41e3-b29d-b8f2aeaac579 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:11:53.971563 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6185d149-d550-41e3-b29d-b8f2aeaac579 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:11:53.971678 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6185d149-d550-41e3-b29d-b8f2aeaac579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:11:53.991016 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6185d149-d550-41e3-b29d-b8f2aeaac579 None None] MFA Rules not processed for user `293c695f15c8437d81a619f840318f44`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:11:53.992250 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6185d149-d550-41e3-b29d-b8f2aeaac579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:11:54.000863 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 685/1355] 10.222.0.71 () {58 vars in 1018 bytes} [Mon Jul 4 09:11:53 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:11:57.594553 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab37138-aa81-445a-8fc2-67680ca20963 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:11:57.595065 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab37138-aa81-445a-8fc2-67680ca20963 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:11:57.595065 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab37138-aa81-445a-8fc2-67680ca20963 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:11:57.596798 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 671/1356] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:11:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:11:57.603487 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ab389-2df1-4d8d-86af-dd9021d72eee None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:11:57.603973 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ab389-2df1-4d8d-86af-dd9021d72eee None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:11:57.604068 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ab389-2df1-4d8d-86af-dd9021d72eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:11:57.624280 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-289ab389-2df1-4d8d-86af-dd9021d72eee None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:11:57.625530 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-289ab389-2df1-4d8d-86af-dd9021d72eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:11:57.634394 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 686/1357] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:11:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:11:57.689672 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fcc237-3af8-4fc3-84fe-e302cd209b76 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:11:57.690317 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fcc237-3af8-4fc3-84fe-e302cd209b76 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:11:57.690829 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fcc237-3af8-4fc3-84fe-e302cd209b76 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:11:57.692398 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 672/1358] 10.222.0.71 () {54 vars in 961 bytes} [Mon Jul 4 09:11:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:11:57.702156 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981ee8c4-7587-4cd2-aba3-9c842a695bab None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:11:57.702663 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981ee8c4-7587-4cd2-aba3-9c842a695bab None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:11:57.703282 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981ee8c4-7587-4cd2-aba3-9c842a695bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:11:57.754868 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-981ee8c4-7587-4cd2-aba3-9c842a695bab None None] MFA Rules not processed for user `15039da209ec445d93d49e2bd6d854eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:11:57.756170 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-981ee8c4-7587-4cd2-aba3-9c842a695bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:11:57.847896 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 687/1359] 10.222.0.71 () {58 vars in 1058 bytes} [Mon Jul 4 09:11:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:11:57.854630 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fcc237-3af8-4fc3-84fe-e302cd209b76 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:11:57.869886 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fcc237-3af8-4fc3-84fe-e302cd209b76 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:11:57.875436 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cffff1-64e5-4c69-ba36-f13f931091e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15039da209ec445d93d49e2bd6d854eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:11:57.876200 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cffff1-64e5-4c69-ba36-f13f931091e2 service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:11:57.876591 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cffff1-64e5-4c69-ba36-f13f931091e2 service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:11:57.876975 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cffff1-64e5-4c69-ba36-f13f931091e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:11:57.886700 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cffff1-64e5-4c69-ba36-f13f931091e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:11:57.888434 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cffff1-64e5-4c69-ba36-f13f931091e2 service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:11:57.899227 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 673/1360] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:11:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:12:00.190376 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e0927b-9896-448a-945e-ad26150e98c9 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:00.191308 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e0927b-9896-448a-945e-ad26150e98c9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:00.191586 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e0927b-9896-448a-945e-ad26150e98c9 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:00.193195 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 688/1361] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:12:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:12:00.198350 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220d9ef-5857-4602-8aaf-8bed074d0a9e None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:00.198786 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220d9ef-5857-4602-8aaf-8bed074d0a9e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:00.198913 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220d9ef-5857-4602-8aaf-8bed074d0a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:00.216779 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-e220d9ef-5857-4602-8aaf-8bed074d0a9e None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:12:00.218282 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e220d9ef-5857-4602-8aaf-8bed074d0a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:12:00.226709 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 674/1362] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:12:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:12:00.271278 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e0927b-9896-448a-945e-ad26150e98c9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:12:00.285570 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e0927b-9896-448a-945e-ad26150e98c9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:12:00.291970 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392eeed4-29cc-44be-bb10-59790c511c01 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15039da209ec445d93d49e2bd6d854eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:12:00.292614 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392eeed4-29cc-44be-bb10-59790c511c01 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:00.292900 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392eeed4-29cc-44be-bb10-59790c511c01 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:00.293212 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392eeed4-29cc-44be-bb10-59790c511c01 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:00.301887 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392eeed4-29cc-44be-bb10-59790c511c01 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:12:00.304207 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392eeed4-29cc-44be-bb10-59790c511c01 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:12:00.314359 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 689/1363] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:12:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:12:02.700199 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f432e44b-f638-45d6-86d2-43e23b7c9785 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:02.701217 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f432e44b-f638-45d6-86d2-43e23b7c9785 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:02.701602 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f432e44b-f638-45d6-86d2-43e23b7c9785 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:02.703503 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 675/1364] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:12:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:12:02.709689 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd2a7b-2628-49ca-8fed-5fbde0b2852a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:02.710143 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd2a7b-2628-49ca-8fed-5fbde0b2852a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:02.710451 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd2a7b-2628-49ca-8fed-5fbde0b2852a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:02.742391 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7afd2a7b-2628-49ca-8fed-5fbde0b2852a None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:12:02.744076 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7afd2a7b-2628-49ca-8fed-5fbde0b2852a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:12:02.752395 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 690/1365] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:12:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:12:02.797561 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e18a649-14d6-4588-a13c-19cc58967efd None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:02.797851 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e18a649-14d6-4588-a13c-19cc58967efd None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:02.798137 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e18a649-14d6-4588-a13c-19cc58967efd None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:02.799503 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 676/1366] 10.222.0.71 () {54 vars in 961 bytes} [Mon Jul 4 09:12:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:12:02.805810 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ffc98-d1ed-4479-834b-981c4aed8699 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:02.806168 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ffc98-d1ed-4479-834b-981c4aed8699 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:02.806516 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ffc98-d1ed-4479-834b-981c4aed8699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:02.824902 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a15ffc98-d1ed-4479-834b-981c4aed8699 None None] MFA Rules not processed for user `15039da209ec445d93d49e2bd6d854eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:12:02.826114 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a15ffc98-d1ed-4479-834b-981c4aed8699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:12:02.835786 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 691/1367] 10.222.0.71 () {58 vars in 1058 bytes} [Mon Jul 4 09:12:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:12:02.843825 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e18a649-14d6-4588-a13c-19cc58967efd None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:12:02.859978 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e18a649-14d6-4588-a13c-19cc58967efd None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:12:02.866274 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870a6186-d5a8-4514-aebf-e5f6fe05f976 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15039da209ec445d93d49e2bd6d854eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:12:02.866916 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a6186-d5a8-4514-aebf-e5f6fe05f976 service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:02.867202 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a6186-d5a8-4514-aebf-e5f6fe05f976 service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:02.867482 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a6186-d5a8-4514-aebf-e5f6fe05f976 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:02.876034 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870a6186-d5a8-4514-aebf-e5f6fe05f976 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:12:02.877653 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870a6186-d5a8-4514-aebf-e5f6fe05f976 service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:12:02.888097 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 677/1368] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:12:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:12:05.007441 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1447215-67f8-4846-b7f7-78f5157ac630 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:05.008000 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1447215-67f8-4846-b7f7-78f5157ac630 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:05.008000 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1447215-67f8-4846-b7f7-78f5157ac630 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:05.009661 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 692/1369] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:12:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:12:05.015104 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e950b-706d-4361-926d-34a34f1f95f6 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:05.015664 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e950b-706d-4361-926d-34a34f1f95f6 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:05.015664 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e950b-706d-4361-926d-34a34f1f95f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:05.035392 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-f74e950b-706d-4361-926d-34a34f1f95f6 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:12:05.036826 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f74e950b-706d-4361-926d-34a34f1f95f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:12:05.049078 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 678/1370] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:12:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:12:05.089758 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1447215-67f8-4846-b7f7-78f5157ac630 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:12:05.106465 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1447215-67f8-4846-b7f7-78f5157ac630 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:12:05.112953 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f023550-a7b7-46ac-a781-1e82cf519721 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15039da209ec445d93d49e2bd6d854eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:12:05.113728 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f023550-a7b7-46ac-a781-1e82cf519721 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:05.114029 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f023550-a7b7-46ac-a781-1e82cf519721 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:05.114306 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f023550-a7b7-46ac-a781-1e82cf519721 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:05.124166 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f023550-a7b7-46ac-a781-1e82cf519721 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:12:05.125876 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f023550-a7b7-46ac-a781-1e82cf519721 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:12:05.136744 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 693/1371] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:12:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:12:05.756478 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9385fb36-4416-48e2-a398-78048a358798 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:05.756856 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9385fb36-4416-48e2-a398-78048a358798 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:05.757175 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9385fb36-4416-48e2-a398-78048a358798 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:05.758574 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 679/1372] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:12:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:12:05.763885 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f27cb-88b5-42da-8dfc-1a45d062b6d6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:05.764181 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f27cb-88b5-42da-8dfc-1a45d062b6d6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:05.764459 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f27cb-88b5-42da-8dfc-1a45d062b6d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:05.782854 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-321f27cb-88b5-42da-8dfc-1a45d062b6d6 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:12:05.784060 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-321f27cb-88b5-42da-8dfc-1a45d062b6d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:12:05.805516 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 694/1373] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:12:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:12:06.148296 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6d1116-ebf9-4c65-9435-8d9266482cf4 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:06.148796 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6d1116-ebf9-4c65-9435-8d9266482cf4 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:06.149019 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6d1116-ebf9-4c65-9435-8d9266482cf4 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:06.150434 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 680/1374] 10.222.0.71 () {54 vars in 963 bytes} [Mon Jul 4 09:12:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:12:06.155549 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326626bd-6b13-4e53-8cdd-3fdb98dd12fc None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:06.155980 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326626bd-6b13-4e53-8cdd-3fdb98dd12fc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:06.156118 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326626bd-6b13-4e53-8cdd-3fdb98dd12fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:06.165082 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:12:06.210331 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-326626bd-6b13-4e53-8cdd-3fdb98dd12fc None None] MFA Rules not processed for user `f6e50a3715a14714b3951d36338dde42`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:12:06.211648 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-326626bd-6b13-4e53-8cdd-3fdb98dd12fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:12:06.277322 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 695/1375] 10.222.0.71 () {58 vars in 1060 bytes} [Mon Jul 4 09:12:06 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:12:06.285587 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6d1116-ebf9-4c65-9435-8d9266482cf4 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:12:06.297632 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6d1116-ebf9-4c65-9435-8d9266482cf4 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:12:06.301186 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05be5b0f-8091-4932-894a-ed4f50846106 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:12:06.302048 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be5b0f-8091-4932-894a-ed4f50846106 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:06.303052 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be5b0f-8091-4932-894a-ed4f50846106 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:06.303340 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be5b0f-8091-4932-894a-ed4f50846106 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:06.314059 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05be5b0f-8091-4932-894a-ed4f50846106 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:12:06.315550 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05be5b0f-8091-4932-894a-ed4f50846106 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:12:06.327651 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 681/1376] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:12:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:12:07.283007 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3187dbc-d068-45b1-945c-8cba54d761dc None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:07.283515 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3187dbc-d068-45b1-945c-8cba54d761dc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:07.283515 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3187dbc-d068-45b1-945c-8cba54d761dc None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:07.285000 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 696/1377] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:12:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:12:07.290790 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a7a34-b634-465b-a687-160d46edd48c None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:07.291209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a7a34-b634-465b-a687-160d46edd48c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:07.291357 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a7a34-b634-465b-a687-160d46edd48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:07.308758 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-598a7a34-b634-465b-a687-160d46edd48c None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:12:07.310019 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-598a7a34-b634-465b-a687-160d46edd48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:12:07.318187 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 682/1378] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:12:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:12:07.363607 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3187dbc-d068-45b1-945c-8cba54d761dc None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:12:07.378101 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3187dbc-d068-45b1-945c-8cba54d761dc None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:12:07.383626 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baff26a0-01df-41ef-bdde-24e04063ea42 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15039da209ec445d93d49e2bd6d854eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:12:07.384276 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff26a0-01df-41ef-bdde-24e04063ea42 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:07.384567 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff26a0-01df-41ef-bdde-24e04063ea42 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:07.384934 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff26a0-01df-41ef-bdde-24e04063ea42 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:07.394290 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baff26a0-01df-41ef-bdde-24e04063ea42 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:12:07.395553 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baff26a0-01df-41ef-bdde-24e04063ea42 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:12:07.406126 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 697/1379] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:12:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:12:09.479249 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec76f2e-2896-482c-9f5c-b986409f49db None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:09.479766 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec76f2e-2896-482c-9f5c-b986409f49db None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:09.479892 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec76f2e-2896-482c-9f5c-b986409f49db None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:09.481297 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 683/1380] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:12:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:12:09.486601 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac474f-a694-46ab-8497-6e0ab1a86e56 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:09.487007 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac474f-a694-46ab-8497-6e0ab1a86e56 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:09.487164 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac474f-a694-46ab-8497-6e0ab1a86e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:09.506829 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ceac474f-a694-46ab-8497-6e0ab1a86e56 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:12:09.508023 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ceac474f-a694-46ab-8497-6e0ab1a86e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:12:09.515871 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 698/1381] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:12:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:12:09.559180 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec76f2e-2896-482c-9f5c-b986409f49db None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:12:09.574151 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec76f2e-2896-482c-9f5c-b986409f49db None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:12:09.579484 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf2c0d8-8761-4e55-9aee-992c521dcfb1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15039da209ec445d93d49e2bd6d854eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:12:09.580116 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf2c0d8-8761-4e55-9aee-992c521dcfb1 service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:09.580405 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf2c0d8-8761-4e55-9aee-992c521dcfb1 service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:09.580684 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf2c0d8-8761-4e55-9aee-992c521dcfb1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:09.588853 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf2c0d8-8761-4e55-9aee-992c521dcfb1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:12:09.591142 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf2c0d8-8761-4e55-9aee-992c521dcfb1 service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:12:09.600529 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 684/1382] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:12:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:12:11.732928 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43d6f7-0a4f-4e6e-a45e-be357ea541f7 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:11.733504 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43d6f7-0a4f-4e6e-a45e-be357ea541f7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:11.733504 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43d6f7-0a4f-4e6e-a45e-be357ea541f7 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:11.734857 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 699/1383] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:12:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:12:11.740713 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadbed44-8a1b-4045-9bf4-163d5be9188d None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:11.741172 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadbed44-8a1b-4045-9bf4-163d5be9188d None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:11.741321 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadbed44-8a1b-4045-9bf4-163d5be9188d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:11.762716 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-aadbed44-8a1b-4045-9bf4-163d5be9188d None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:12:11.763934 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-aadbed44-8a1b-4045-9bf4-163d5be9188d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:12:11.773429 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 685/1384] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:12:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:12:11.815753 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec43d6f7-0a4f-4e6e-a45e-be357ea541f7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:12:11.833561 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec43d6f7-0a4f-4e6e-a45e-be357ea541f7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:12:11.840152 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2401ae38-af80-4cc3-9d29-84717fd83aaf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15039da209ec445d93d49e2bd6d854eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:12:11.841299 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2401ae38-af80-4cc3-9d29-84717fd83aaf service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:11.841797 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2401ae38-af80-4cc3-9d29-84717fd83aaf service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:11.842298 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2401ae38-af80-4cc3-9d29-84717fd83aaf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:11.852647 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2401ae38-af80-4cc3-9d29-84717fd83aaf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:12:11.854886 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2401ae38-af80-4cc3-9d29-84717fd83aaf service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:12:11.866095 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 700/1385] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:12:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:12:13.960796 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea8f56-bcf7-4fca-a4f0-362af9c923ca None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:13.961383 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea8f56-bcf7-4fca-a4f0-362af9c923ca None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:13.961383 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea8f56-bcf7-4fca-a4f0-362af9c923ca None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:13.962763 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 686/1386] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:12:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:12:13.968333 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc93148-bfd6-4a72-89cc-b10c9328ddef None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:13.968823 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc93148-bfd6-4a72-89cc-b10c9328ddef None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:13.969023 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc93148-bfd6-4a72-89cc-b10c9328ddef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:13.987657 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-fbc93148-bfd6-4a72-89cc-b10c9328ddef None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:12:13.988842 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fbc93148-bfd6-4a72-89cc-b10c9328ddef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:12:13.998219 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 701/1387] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:12:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:12:14.044465 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ea8f56-bcf7-4fca-a4f0-362af9c923ca None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:12:14.061917 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ea8f56-bcf7-4fca-a4f0-362af9c923ca None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:12:14.073634 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aa9755-0dce-4932-980a-e75eac0f412f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15039da209ec445d93d49e2bd6d854eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:12:14.074616 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa9755-0dce-4932-980a-e75eac0f412f service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:14.075005 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa9755-0dce-4932-980a-e75eac0f412f service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:14.075290 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa9755-0dce-4932-980a-e75eac0f412f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:14.085368 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aa9755-0dce-4932-980a-e75eac0f412f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:12:14.087972 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aa9755-0dce-4932-980a-e75eac0f412f service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:12:14.105150 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 687/1388] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:12:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:12:16.259904 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a411d8c-0128-4fbf-bfd1-f1e7b6fd0240 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:16.260461 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a411d8c-0128-4fbf-bfd1-f1e7b6fd0240 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:16.260555 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a411d8c-0128-4fbf-bfd1-f1e7b6fd0240 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:16.261952 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 702/1389] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:12:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:12:16.267261 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c1633-090f-4135-bfcf-b44805698b21 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:16.267718 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c1633-090f-4135-bfcf-b44805698b21 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:16.267813 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c1633-090f-4135-bfcf-b44805698b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:16.287154 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-612c1633-090f-4135-bfcf-b44805698b21 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:12:16.288359 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-612c1633-090f-4135-bfcf-b44805698b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:12:16.296321 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 688/1390] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:12:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:12:16.351420 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a411d8c-0128-4fbf-bfd1-f1e7b6fd0240 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:12:16.369973 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a411d8c-0128-4fbf-bfd1-f1e7b6fd0240 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:12:16.376769 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ad2327-9e86-40d8-9d0d-abf6b259393e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15039da209ec445d93d49e2bd6d854eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:12:16.377656 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ad2327-9e86-40d8-9d0d-abf6b259393e service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:16.378020 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ad2327-9e86-40d8-9d0d-abf6b259393e service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:16.378379 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ad2327-9e86-40d8-9d0d-abf6b259393e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:16.389414 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ad2327-9e86-40d8-9d0d-abf6b259393e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:12:16.391768 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ad2327-9e86-40d8-9d0d-abf6b259393e service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:12:16.404826 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 703/1391] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:12:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:12:19.000908 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012eae0e-edec-4fc4-a194-ab09394a6003 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:19.001524 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012eae0e-edec-4fc4-a194-ab09394a6003 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:19.001524 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012eae0e-edec-4fc4-a194-ab09394a6003 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:19.002882 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 689/1392] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:12:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:12:19.010585 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4103d8ff-64fc-4239-bf11-dcc3e9afc877 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:19.011050 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4103d8ff-64fc-4239-bf11-dcc3e9afc877 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:19.011296 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4103d8ff-64fc-4239-bf11-dcc3e9afc877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:19.034233 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4103d8ff-64fc-4239-bf11-dcc3e9afc877 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:12:19.035731 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4103d8ff-64fc-4239-bf11-dcc3e9afc877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:12:19.046276 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 704/1393] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:12:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:12:19.103166 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012eae0e-edec-4fc4-a194-ab09394a6003 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:12:19.117808 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012eae0e-edec-4fc4-a194-ab09394a6003 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:12:19.123202 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fbf82e-0d93-4d79-99b2-e45219375a0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15039da209ec445d93d49e2bd6d854eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:12:19.123907 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fbf82e-0d93-4d79-99b2-e45219375a0b service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:19.124134 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fbf82e-0d93-4d79-99b2-e45219375a0b service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:19.124438 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fbf82e-0d93-4d79-99b2-e45219375a0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:19.133724 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fbf82e-0d93-4d79-99b2-e45219375a0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:12:19.135311 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fbf82e-0d93-4d79-99b2-e45219375a0b service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:12:19.145178 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 690/1394] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:12:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:12:21.197418 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc19eb-c958-4c2f-b258-0d230eb01b38 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:21.198000 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc19eb-c958-4c2f-b258-0d230eb01b38 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:21.198000 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc19eb-c958-4c2f-b258-0d230eb01b38 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:21.199399 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 705/1395] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:12:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:12:21.204957 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce93e393-1410-4520-92d3-10898bc13ff9 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:21.205519 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce93e393-1410-4520-92d3-10898bc13ff9 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:21.205519 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce93e393-1410-4520-92d3-10898bc13ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:21.226139 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-ce93e393-1410-4520-92d3-10898bc13ff9 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:12:21.227329 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ce93e393-1410-4520-92d3-10898bc13ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:12:21.235469 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 691/1396] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:12:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:12:21.280655 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc19eb-c958-4c2f-b258-0d230eb01b38 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:12:21.295227 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc19eb-c958-4c2f-b258-0d230eb01b38 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:12:21.300595 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6966b391-dd93-473d-826f-8fb793f601bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15039da209ec445d93d49e2bd6d854eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:12:21.301276 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6966b391-dd93-473d-826f-8fb793f601bb service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:21.301563 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6966b391-dd93-473d-826f-8fb793f601bb service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:21.301838 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6966b391-dd93-473d-826f-8fb793f601bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:21.310222 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6966b391-dd93-473d-826f-8fb793f601bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:12:21.311785 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6966b391-dd93-473d-826f-8fb793f601bb service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:12:21.321868 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 706/1397] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:12:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:12:21.576512 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a039eae-dba5-4f05-a358-e586345f7cf8 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:21.576803 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a039eae-dba5-4f05-a358-e586345f7cf8 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:21.577117 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a039eae-dba5-4f05-a358-e586345f7cf8 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:21.578545 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 692/1398] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:12:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:12:21.584639 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32766ac9-e7e5-41cf-905f-65b88b3897c8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:21.585137 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32766ac9-e7e5-41cf-905f-65b88b3897c8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:21.585431 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32766ac9-e7e5-41cf-905f-65b88b3897c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:21.604173 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-32766ac9-e7e5-41cf-905f-65b88b3897c8 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:12:21.605408 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-32766ac9-e7e5-41cf-905f-65b88b3897c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:12:21.613190 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 707/1399] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:12:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:12:21.669129 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10ca02-77fc-4270-bdc3-f70a94bf4bcd None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:21.669424 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10ca02-77fc-4270-bdc3-f70a94bf4bcd None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:21.669713 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10ca02-77fc-4270-bdc3-f70a94bf4bcd None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:21.671081 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 693/1400] 10.222.0.71 () {54 vars in 962 bytes} [Mon Jul 4 09:12:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:12:21.676060 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d00ce-161b-4e62-88af-529e80f6f66f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:21.676351 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d00ce-161b-4e62-88af-529e80f6f66f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:21.676637 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d00ce-161b-4e62-88af-529e80f6f66f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:21.727115 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-da8d00ce-161b-4e62-88af-529e80f6f66f None None] MFA Rules not processed for user `192724527d774817bc4581474ac848dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:12:21.728326 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-da8d00ce-161b-4e62-88af-529e80f6f66f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:12:21.760994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:12:21.791454 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 708/1401] 10.222.0.71 () {58 vars in 1059 bytes} [Mon Jul 4 09:12:21 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:12:21.798986 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d10ca02-77fc-4270-bdc3-f70a94bf4bcd None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:12:21.813679 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d10ca02-77fc-4270-bdc3-f70a94bf4bcd None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:12:21.827355 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aec985-3edd-4cd7-af27-bce4b4ed1e29 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '192724527d774817bc4581474ac848dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:12:21.827718 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aec985-3edd-4cd7-af27-bce4b4ed1e29 service glance] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:21.827919 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aec985-3edd-4cd7-af27-bce4b4ed1e29 service glance] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:21.828081 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aec985-3edd-4cd7-af27-bce4b4ed1e29 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:21.838993 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aec985-3edd-4cd7-af27-bce4b4ed1e29 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:12:21.840439 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aec985-3edd-4cd7-af27-bce4b4ed1e29 service glance] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:12:21.851156 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 694/1402] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:12:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:12:22.588765 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e0ff4-97f8-4032-9388-abd91987c810 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:22.589323 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e0ff4-97f8-4032-9388-abd91987c810 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:22.589431 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e0ff4-97f8-4032-9388-abd91987c810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:22.638922 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6e4e0ff4-97f8-4032-9388-abd91987c810 None None] MFA Rules not processed for user `78668b939c5847b1b73e9c28b5d8db7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:12:22.640136 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6e4e0ff4-97f8-4032-9388-abd91987c810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:12:22.704652 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 709/1403] 10.222.0.71 () {58 vars in 1009 bytes} [Mon Jul 4 09:12:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:12:22.749019 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c34aec3-c9e8-462a-be09-ca56f32e0e56 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:22.749509 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c34aec3-c9e8-462a-be09-ca56f32e0e56 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:22.749753 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c34aec3-c9e8-462a-be09-ca56f32e0e56 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:22.751148 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 695/1404] 10.222.0.71 () {54 vars in 942 bytes} [Mon Jul 4 09:12:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:12:22.757744 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d4e06e-cab9-464b-8707-5f4f1f9a1fa2 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:22.758041 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d4e06e-cab9-464b-8707-5f4f1f9a1fa2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:22.758329 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d4e06e-cab9-464b-8707-5f4f1f9a1fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:22.805730 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-13d4e06e-cab9-464b-8707-5f4f1f9a1fa2 None None] MFA Rules not processed for user `88b41b55769a4c449716c3590ed58225`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:12:22.806916 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-13d4e06e-cab9-464b-8707-5f4f1f9a1fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:12:22.877969 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 710/1405] 10.222.0.71 () {58 vars in 1039 bytes} [Mon Jul 4 09:12:22 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:12:22.888315 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c34aec3-c9e8-462a-be09-ca56f32e0e56 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:12:22.895672 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=61 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:12:22.904440 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c34aec3-c9e8-462a-be09-ca56f32e0e56 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:12:22.909513 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6642d4ae-5c29-4679-9b77-9ca6f51185a9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88b41b55769a4c449716c3590ed58225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:12:22.910186 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6642d4ae-5c29-4679-9b77-9ca6f51185a9 service swift] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:22.910468 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6642d4ae-5c29-4679-9b77-9ca6f51185a9 service swift] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:22.910744 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6642d4ae-5c29-4679-9b77-9ca6f51185a9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:22.919633 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6642d4ae-5c29-4679-9b77-9ca6f51185a9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:12:22.921250 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6642d4ae-5c29-4679-9b77-9ca6f51185a9 service swift] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:12:22.929832 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 696/1406] 10.222.0.71 () {60 vars in 1384 bytes} [Mon Jul 4 09:12:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 04 09:12:23.422073 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc55715-3878-4939-b5fe-d39e82e95f22 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:23.422534 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc55715-3878-4939-b5fe-d39e82e95f22 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:23.422888 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc55715-3878-4939-b5fe-d39e82e95f22 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:23.424477 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 711/1407] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:12:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:12:23.430468 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b2e699-c2e5-4302-a1d1-6dd6fbeaf690 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:23.430872 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b2e699-c2e5-4302-a1d1-6dd6fbeaf690 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:23.431269 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b2e699-c2e5-4302-a1d1-6dd6fbeaf690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:23.453094 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-27b2e699-c2e5-4302-a1d1-6dd6fbeaf690 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:12:23.454421 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-27b2e699-c2e5-4302-a1d1-6dd6fbeaf690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:12:23.466256 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 697/1408] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:12:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:12:23.533661 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc55715-3878-4939-b5fe-d39e82e95f22 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:12:23.551553 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc55715-3878-4939-b5fe-d39e82e95f22 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:12:23.558364 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f315af4-f927-4a9c-a388-151d60a186d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15039da209ec445d93d49e2bd6d854eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:12:23.559016 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f315af4-f927-4a9c-a388-151d60a186d6 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:23.559297 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f315af4-f927-4a9c-a388-151d60a186d6 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:23.559572 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f315af4-f927-4a9c-a388-151d60a186d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:23.570837 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f315af4-f927-4a9c-a388-151d60a186d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:12:23.572474 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f315af4-f927-4a9c-a388-151d60a186d6 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:12:23.577915 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:12:23.584479 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 712/1409] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:12:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:12:33.447684 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=8 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:12:45.299545 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb5b258-d74c-4e66-8c05-3dbe38180ca7 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:45.300228 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb5b258-d74c-4e66-8c05-3dbe38180ca7 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:45.300228 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb5b258-d74c-4e66-8c05-3dbe38180ca7 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:45.301572 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 698/1410] 10.222.0.71 () {54 vars in 918 bytes} [Mon Jul 4 09:12:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:12:45.307390 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea72db-aecb-4e63-a0f9-e9e82af26469 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:45.307807 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea72db-aecb-4e63-a0f9-e9e82af26469 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:45.307968 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea72db-aecb-4e63-a0f9-e9e82af26469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:45.325733 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-11ea72db-aecb-4e63-a0f9-e9e82af26469 None None] MFA Rules not processed for user `293c695f15c8437d81a619f840318f44`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:12:45.326984 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-11ea72db-aecb-4e63-a0f9-e9e82af26469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:12:45.335523 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 713/1411] 10.222.0.71 () {58 vars in 1015 bytes} [Mon Jul 4 09:12:45 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:12:45.350981 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338dea05-e2bb-4b0b-8893-8f2cce5fb4c1 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:45.351339 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338dea05-e2bb-4b0b-8893-8f2cce5fb4c1 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:45.351693 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338dea05-e2bb-4b0b-8893-8f2cce5fb4c1 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:45.353164 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 699/1412] 10.222.0.71 () {54 vars in 918 bytes} [Mon Jul 4 09:12:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:12:45.357838 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf91bf6-e6c8-4587-9198-7a6cd360d8ca None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:45.358131 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf91bf6-e6c8-4587-9198-7a6cd360d8ca None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:45.358412 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf91bf6-e6c8-4587-9198-7a6cd360d8ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:45.376898 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7cf91bf6-e6c8-4587-9198-7a6cd360d8ca None None] MFA Rules not processed for user `15039da209ec445d93d49e2bd6d854eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:12:45.378149 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7cf91bf6-e6c8-4587-9198-7a6cd360d8ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:12:45.388055 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 714/1413] 10.222.0.71 () {58 vars in 1015 bytes} [Mon Jul 4 09:12:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:12:47.915129 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db61be-cec5-4050-b109-09bc370eadff None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:47.915700 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db61be-cec5-4050-b109-09bc370eadff None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:47.915700 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db61be-cec5-4050-b109-09bc370eadff None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:47.917147 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 700/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 09:12:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:12:47.922423 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167d5e1-eaf7-40f0-b469-6749b368de92 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:47.922838 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167d5e1-eaf7-40f0-b469-6749b368de92 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:47.923025 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167d5e1-eaf7-40f0-b469-6749b368de92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:47.940203 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7167d5e1-eaf7-40f0-b469-6749b368de92 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:12:47.941447 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7167d5e1-eaf7-40f0-b469-6749b368de92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:12:47.949012 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 715/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 09:12:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:12:47.992540 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87db61be-cec5-4050-b109-09bc370eadff None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:12:48.007448 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87db61be-cec5-4050-b109-09bc370eadff None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:12:48.012824 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa88b436-ec34-406f-809d-6ca82b5e23f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15039da209ec445d93d49e2bd6d854eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:12:48.013494 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa88b436-ec34-406f-809d-6ca82b5e23f0 service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:48.013785 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa88b436-ec34-406f-809d-6ca82b5e23f0 service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:48.014060 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa88b436-ec34-406f-809d-6ca82b5e23f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:48.022674 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa88b436-ec34-406f-809d-6ca82b5e23f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:12:48.024325 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa88b436-ec34-406f-809d-6ca82b5e23f0 service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:12:48.033749 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 701/1416] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:12:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:12:48.936085 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e911084-594e-439b-8e0e-40177e4ce231 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:48.936660 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e911084-594e-439b-8e0e-40177e4ce231 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:48.936660 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e911084-594e-439b-8e0e-40177e4ce231 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:48.938183 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 716/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 09:12:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:12:48.943758 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b9cd4-8e4c-49e2-b732-d980158db2c0 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:48.944202 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b9cd4-8e4c-49e2-b732-d980158db2c0 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:48.944346 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b9cd4-8e4c-49e2-b732-d980158db2c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:48.962973 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-819b9cd4-8e4c-49e2-b732-d980158db2c0 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:12:48.964164 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-819b9cd4-8e4c-49e2-b732-d980158db2c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:12:48.972138 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 702/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 09:12:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:12:49.127023 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e911084-594e-439b-8e0e-40177e4ce231 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:12:49.142131 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e911084-594e-439b-8e0e-40177e4ce231 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:12:49.148212 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf07f5b-5e73-4f6f-98fa-0ddfb7afbce3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15039da209ec445d93d49e2bd6d854eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:12:49.148850 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf07f5b-5e73-4f6f-98fa-0ddfb7afbce3 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:49.149169 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf07f5b-5e73-4f6f-98fa-0ddfb7afbce3 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:49.149454 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf07f5b-5e73-4f6f-98fa-0ddfb7afbce3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:49.157441 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf07f5b-5e73-4f6f-98fa-0ddfb7afbce3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:12:49.159028 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf07f5b-5e73-4f6f-98fa-0ddfb7afbce3 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:12:49.168464 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 717/1419] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:12:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:12:50.046972 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bffcf05-ff8f-40c0-bb2c-79bfee0857a5 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:50.047575 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bffcf05-ff8f-40c0-bb2c-79bfee0857a5 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:50.047575 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bffcf05-ff8f-40c0-bb2c-79bfee0857a5 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:50.048904 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 703/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 09:12:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:12:50.054023 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d39c8e-2213-4fdb-b234-0efd20823973 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:50.054499 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d39c8e-2213-4fdb-b234-0efd20823973 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:50.054823 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d39c8e-2213-4fdb-b234-0efd20823973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:50.072776 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d7d39c8e-2213-4fdb-b234-0efd20823973 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:12:50.074021 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d7d39c8e-2213-4fdb-b234-0efd20823973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:12:50.081785 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 718/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 09:12:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:12:50.233292 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bffcf05-ff8f-40c0-bb2c-79bfee0857a5 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:12:50.247392 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bffcf05-ff8f-40c0-bb2c-79bfee0857a5 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:12:50.252686 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69739958-a84a-4eb3-a423-424c13f79ea7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15039da209ec445d93d49e2bd6d854eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:12:50.253358 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69739958-a84a-4eb3-a423-424c13f79ea7 service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:50.253644 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69739958-a84a-4eb3-a423-424c13f79ea7 service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:50.253925 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69739958-a84a-4eb3-a423-424c13f79ea7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:50.262186 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69739958-a84a-4eb3-a423-424c13f79ea7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:12:50.263767 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69739958-a84a-4eb3-a423-424c13f79ea7 service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:12:50.273804 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 704/1422] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:12:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:12:51.152255 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b68c02-4a3e-4b7c-871a-d3937d324975 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:51.152805 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b68c02-4a3e-4b7c-871a-d3937d324975 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:51.152805 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b68c02-4a3e-4b7c-871a-d3937d324975 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:51.154211 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 719/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 09:12:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:12:51.158997 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c185db-2044-4b75-9db8-4a1a5cd7b4a3 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:51.159412 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c185db-2044-4b75-9db8-4a1a5cd7b4a3 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:51.159630 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c185db-2044-4b75-9db8-4a1a5cd7b4a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:51.178529 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-84c185db-2044-4b75-9db8-4a1a5cd7b4a3 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:12:51.179724 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-84c185db-2044-4b75-9db8-4a1a5cd7b4a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:12:51.187476 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 705/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 09:12:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:12:51.337757 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b68c02-4a3e-4b7c-871a-d3937d324975 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:12:51.352964 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b68c02-4a3e-4b7c-871a-d3937d324975 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:12:51.358558 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feefdd7a-bfc1-4cc7-987b-c995518ecc57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15039da209ec445d93d49e2bd6d854eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:12:51.359197 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feefdd7a-bfc1-4cc7-987b-c995518ecc57 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:51.359482 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feefdd7a-bfc1-4cc7-987b-c995518ecc57 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:51.359761 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feefdd7a-bfc1-4cc7-987b-c995518ecc57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:51.368508 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feefdd7a-bfc1-4cc7-987b-c995518ecc57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:12:51.370092 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feefdd7a-bfc1-4cc7-987b-c995518ecc57 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:12:51.379306 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 720/1425] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:12:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:12:52.231640 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d3e83d-fb5d-46b2-9353-36f20464e6bf None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:52.232208 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d3e83d-fb5d-46b2-9353-36f20464e6bf None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:52.232208 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d3e83d-fb5d-46b2-9353-36f20464e6bf None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:52.233621 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 706/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 09:12:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:12:52.238471 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337f648-1941-4cad-9204-3463a2d04bdb None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:52.238914 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337f648-1941-4cad-9204-3463a2d04bdb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:52.239030 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337f648-1941-4cad-9204-3463a2d04bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:52.257798 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d337f648-1941-4cad-9204-3463a2d04bdb None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:12:52.258987 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d337f648-1941-4cad-9204-3463a2d04bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:12:52.266696 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 721/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 09:12:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:12:52.417688 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d3e83d-fb5d-46b2-9353-36f20464e6bf None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:12:52.432567 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d3e83d-fb5d-46b2-9353-36f20464e6bf None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:12:52.438154 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6886f1-0576-4768-bab3-d80a6acd280e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15039da209ec445d93d49e2bd6d854eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:12:52.438824 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6886f1-0576-4768-bab3-d80a6acd280e service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:52.439117 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6886f1-0576-4768-bab3-d80a6acd280e service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:52.439425 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6886f1-0576-4768-bab3-d80a6acd280e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:52.448545 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6886f1-0576-4768-bab3-d80a6acd280e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:12:52.450166 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6886f1-0576-4768-bab3-d80a6acd280e service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:12:52.459761 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 707/1428] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:12:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:12:53.261756 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7635dff-23e7-4bc3-abfd-ef481798885f None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:53.262348 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7635dff-23e7-4bc3-abfd-ef481798885f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:53.262348 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7635dff-23e7-4bc3-abfd-ef481798885f None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:53.263741 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 722/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 09:12:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:12:53.268686 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdde71d-2e77-433d-894b-8b19190ebeb5 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:53.269129 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdde71d-2e77-433d-894b-8b19190ebeb5 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:53.269279 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdde71d-2e77-433d-894b-8b19190ebeb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:53.288068 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-9cdde71d-2e77-433d-894b-8b19190ebeb5 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:12:53.289305 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-9cdde71d-2e77-433d-894b-8b19190ebeb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:12:53.297214 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 708/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 09:12:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:12:53.449459 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7635dff-23e7-4bc3-abfd-ef481798885f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:12:53.464506 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7635dff-23e7-4bc3-abfd-ef481798885f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:12:53.470096 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab79a145-0c60-4e26-ab30-9ae63426d681 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15039da209ec445d93d49e2bd6d854eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:12:53.470734 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab79a145-0c60-4e26-ab30-9ae63426d681 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:53.471013 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab79a145-0c60-4e26-ab30-9ae63426d681 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:53.471287 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab79a145-0c60-4e26-ab30-9ae63426d681 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:53.479784 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab79a145-0c60-4e26-ab30-9ae63426d681 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:12:53.481391 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab79a145-0c60-4e26-ab30-9ae63426d681 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:12:53.490993 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 723/1431] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:12:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:12:54.298187 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b576b3bf-1914-4b55-a942-0f7dc1d89ed4 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:54.298756 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b576b3bf-1914-4b55-a942-0f7dc1d89ed4 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:54.298756 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b576b3bf-1914-4b55-a942-0f7dc1d89ed4 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:54.300206 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 709/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 09:12:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:12:54.305833 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac293f3-91f1-4fc6-84c8-4476edf8246f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:54.306273 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac293f3-91f1-4fc6-84c8-4476edf8246f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:54.306375 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac293f3-91f1-4fc6-84c8-4476edf8246f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:54.324566 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6ac293f3-91f1-4fc6-84c8-4476edf8246f None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:12:54.325863 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6ac293f3-91f1-4fc6-84c8-4476edf8246f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:12:54.334449 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 724/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 09:12:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:12:54.487466 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b576b3bf-1914-4b55-a942-0f7dc1d89ed4 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:12:54.502520 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b576b3bf-1914-4b55-a942-0f7dc1d89ed4 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:12:54.508593 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4470b7a6-1d12-4ef9-9f52-745802c8129a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15039da209ec445d93d49e2bd6d854eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:12:54.509268 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4470b7a6-1d12-4ef9-9f52-745802c8129a service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:54.509560 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4470b7a6-1d12-4ef9-9f52-745802c8129a service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:54.509847 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4470b7a6-1d12-4ef9-9f52-745802c8129a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:54.517966 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4470b7a6-1d12-4ef9-9f52-745802c8129a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:12:54.519561 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4470b7a6-1d12-4ef9-9f52-745802c8129a service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:12:54.528973 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 710/1434] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:12:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:12:55.334563 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590c7e79-67e6-488a-ad61-bac6af4beff2 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:55.335247 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590c7e79-67e6-488a-ad61-bac6af4beff2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:55.335247 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590c7e79-67e6-488a-ad61-bac6af4beff2 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:55.336489 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 725/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 09:12:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:12:55.341377 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1695d01-6b5b-488e-9bd2-6a42cf24960b None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:55.341826 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1695d01-6b5b-488e-9bd2-6a42cf24960b None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:55.342118 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1695d01-6b5b-488e-9bd2-6a42cf24960b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:55.360717 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-b1695d01-6b5b-488e-9bd2-6a42cf24960b None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:12:55.361963 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b1695d01-6b5b-488e-9bd2-6a42cf24960b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:12:55.370030 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 711/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 09:12:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:12:55.524637 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590c7e79-67e6-488a-ad61-bac6af4beff2 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:12:55.539830 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590c7e79-67e6-488a-ad61-bac6af4beff2 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:12:55.545260 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a95c1-1665-4b09-8650-2584bfd648a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15039da209ec445d93d49e2bd6d854eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:12:55.545900 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a95c1-1665-4b09-8650-2584bfd648a6 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:55.546188 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a95c1-1665-4b09-8650-2584bfd648a6 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:55.546469 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a95c1-1665-4b09-8650-2584bfd648a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:55.554677 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48a95c1-1665-4b09-8650-2584bfd648a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:12:55.556270 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48a95c1-1665-4b09-8650-2584bfd648a6 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:12:55.565612 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 726/1437] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:12:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:12:56.367360 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f4464-9e75-4541-b47b-156c4a855915 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:56.367935 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f4464-9e75-4541-b47b-156c4a855915 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:56.367935 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f4464-9e75-4541-b47b-156c4a855915 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:56.369408 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 712/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 09:12:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:12:56.374177 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf44b4e-ce12-43a5-850f-ec4b26cdb987 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:56.374640 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf44b4e-ce12-43a5-850f-ec4b26cdb987 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:56.374748 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf44b4e-ce12-43a5-850f-ec4b26cdb987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:56.394606 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ecf44b4e-ce12-43a5-850f-ec4b26cdb987 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:12:56.395272 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ecf44b4e-ce12-43a5-850f-ec4b26cdb987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:12:56.407034 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 727/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 09:12:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:12:56.586999 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2f4464-9e75-4541-b47b-156c4a855915 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:12:56.601726 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2f4464-9e75-4541-b47b-156c4a855915 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:12:56.607285 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427865ff-8d76-4a78-9ec9-900db2d2297f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15039da209ec445d93d49e2bd6d854eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:12:56.607931 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427865ff-8d76-4a78-9ec9-900db2d2297f service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:56.608218 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427865ff-8d76-4a78-9ec9-900db2d2297f service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:56.608497 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427865ff-8d76-4a78-9ec9-900db2d2297f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:56.617442 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427865ff-8d76-4a78-9ec9-900db2d2297f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:12:56.619039 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427865ff-8d76-4a78-9ec9-900db2d2297f service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:12:56.629141 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 713/1440] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:12:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:12:58.599710 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c0a22f-257b-45d4-a174-ce48efe46e28 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:58.600251 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c0a22f-257b-45d4-a174-ce48efe46e28 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:58.600376 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c0a22f-257b-45d4-a174-ce48efe46e28 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:58.601735 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 728/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 09:12:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:12:58.606876 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d532c0-7f1d-4f5a-8b21-a7da749cd32c None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:58.607289 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d532c0-7f1d-4f5a-8b21-a7da749cd32c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:58.607443 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d532c0-7f1d-4f5a-8b21-a7da749cd32c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:58.625443 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-d0d532c0-7f1d-4f5a-8b21-a7da749cd32c None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:12:58.626643 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d0d532c0-7f1d-4f5a-8b21-a7da749cd32c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:12:58.634539 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 714/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 09:12:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:12:58.672478 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c0a22f-257b-45d4-a174-ce48efe46e28 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:12:58.684057 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c0a22f-257b-45d4-a174-ce48efe46e28 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:12:58.687655 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac23d972-b623-4692-b723-5c18797e7262 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:12:58.688300 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac23d972-b623-4692-b723-5c18797e7262 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:58.688589 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac23d972-b623-4692-b723-5c18797e7262 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:58.688866 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac23d972-b623-4692-b723-5c18797e7262 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:58.697839 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac23d972-b623-4692-b723-5c18797e7262 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:12:58.699267 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac23d972-b623-4692-b723-5c18797e7262 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:12:58.708636 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 729/1443] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:12:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:13:01.022272 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da767dc2-0690-4412-85ff-65240b094e7f None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:01.022840 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da767dc2-0690-4412-85ff-65240b094e7f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:01.022840 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da767dc2-0690-4412-85ff-65240b094e7f None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:01.024226 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 715/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 09:13:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:13:01.029667 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268ca06b-6872-497c-a9f0-d56a8e4541c1 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:01.030109 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268ca06b-6872-497c-a9f0-d56a8e4541c1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:01.030214 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268ca06b-6872-497c-a9f0-d56a8e4541c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:01.048083 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-268ca06b-6872-497c-a9f0-d56a8e4541c1 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:13:01.049324 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-268ca06b-6872-497c-a9f0-d56a8e4541c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:13:01.056914 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 730/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 09:13:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:13:01.095384 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da767dc2-0690-4412-85ff-65240b094e7f None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:13:01.106165 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da767dc2-0690-4412-85ff-65240b094e7f None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:13:01.109684 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6df195-fbce-43bf-9e1f-9961d2ab5175 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:13:01.110319 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6df195-fbce-43bf-9e1f-9961d2ab5175 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:01.110606 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6df195-fbce-43bf-9e1f-9961d2ab5175 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:01.110890 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6df195-fbce-43bf-9e1f-9961d2ab5175 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:01.119879 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6df195-fbce-43bf-9e1f-9961d2ab5175 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:13:01.121402 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6df195-fbce-43bf-9e1f-9961d2ab5175 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:13:01.131305 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 716/1446] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:13:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:13:03.480104 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c78763-a11f-4efc-a04d-f39e703e2f65 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:03.480896 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c78763-a11f-4efc-a04d-f39e703e2f65 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:03.480896 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c78763-a11f-4efc-a04d-f39e703e2f65 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:03.482069 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 731/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 09:13:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:13:03.487244 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b74066-17e4-471e-b440-52c4e82b2f04 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:03.487696 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b74066-17e4-471e-b440-52c4e82b2f04 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:03.488001 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b74066-17e4-471e-b440-52c4e82b2f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:03.507085 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-97b74066-17e4-471e-b440-52c4e82b2f04 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:13:03.508330 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-97b74066-17e4-471e-b440-52c4e82b2f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:13:03.516303 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 717/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 09:13:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:13:03.555531 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c78763-a11f-4efc-a04d-f39e703e2f65 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:13:03.566219 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c78763-a11f-4efc-a04d-f39e703e2f65 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:13:03.569902 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01964c4-7486-4afc-a9d1-99ce30522a84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:13:03.570535 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01964c4-7486-4afc-a9d1-99ce30522a84 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:03.570818 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01964c4-7486-4afc-a9d1-99ce30522a84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:03.571096 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01964c4-7486-4afc-a9d1-99ce30522a84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:03.580013 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01964c4-7486-4afc-a9d1-99ce30522a84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:13:03.581478 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01964c4-7486-4afc-a9d1-99ce30522a84 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:13:03.591400 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 732/1449] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:13:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:13:05.760111 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef74d4-e6c3-43f4-9ccd-ae6479a697de None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:05.760681 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef74d4-e6c3-43f4-9ccd-ae6479a697de None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:05.760681 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef74d4-e6c3-43f4-9ccd-ae6479a697de None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:05.762089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 718/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 09:13:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:13:05.767274 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bff1e72-1584-4dd6-91f8-1517f1995fff None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:05.767683 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bff1e72-1584-4dd6-91f8-1517f1995fff None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:05.767942 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bff1e72-1584-4dd6-91f8-1517f1995fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:05.785802 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7bff1e72-1584-4dd6-91f8-1517f1995fff None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:13:05.787203 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7bff1e72-1584-4dd6-91f8-1517f1995fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:13:05.795011 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 733/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 09:13:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:13:05.835428 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ef74d4-e6c3-43f4-9ccd-ae6479a697de None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:13:05.846301 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ef74d4-e6c3-43f4-9ccd-ae6479a697de None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:13:05.849952 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2287e601-828d-4a40-b04d-ed4282bc50ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:13:05.850595 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2287e601-828d-4a40-b04d-ed4282bc50ca service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:05.850883 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2287e601-828d-4a40-b04d-ed4282bc50ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:05.851164 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2287e601-828d-4a40-b04d-ed4282bc50ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:05.860536 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2287e601-828d-4a40-b04d-ed4282bc50ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:13:05.862004 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2287e601-828d-4a40-b04d-ed4282bc50ca service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:13:05.872111 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 719/1452] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:13:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:13:07.957510 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f644d6-4a4e-4a05-9b4a-eb745e7ff461 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:07.958130 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f644d6-4a4e-4a05-9b4a-eb745e7ff461 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:07.959033 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f644d6-4a4e-4a05-9b4a-eb745e7ff461 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:07.960393 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 734/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 09:13:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:13:07.965813 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb48af-1536-4fff-8b87-ceab1a260bde None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:07.966239 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb48af-1536-4fff-8b87-ceab1a260bde None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:07.966381 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb48af-1536-4fff-8b87-ceab1a260bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:07.984102 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-c8eb48af-1536-4fff-8b87-ceab1a260bde None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:13:07.985386 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c8eb48af-1536-4fff-8b87-ceab1a260bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:13:07.993919 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 720/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 09:13:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:13:08.031440 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f644d6-4a4e-4a05-9b4a-eb745e7ff461 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:13:08.042513 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f644d6-4a4e-4a05-9b4a-eb745e7ff461 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:13:08.046172 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d9c6ed-9278-4270-b7e8-44fe6e227eec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:13:08.046794 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d9c6ed-9278-4270-b7e8-44fe6e227eec service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:08.047074 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d9c6ed-9278-4270-b7e8-44fe6e227eec service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:08.047344 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d9c6ed-9278-4270-b7e8-44fe6e227eec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:08.056896 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d9c6ed-9278-4270-b7e8-44fe6e227eec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:13:08.058339 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d9c6ed-9278-4270-b7e8-44fe6e227eec service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:13:08.068603 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 735/1455] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:13:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:13:10.177416 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e14b1-d50b-4b77-b4d9-05e62f0aa6b3 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:10.177912 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e14b1-d50b-4b77-b4d9-05e62f0aa6b3 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:10.178007 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e14b1-d50b-4b77-b4d9-05e62f0aa6b3 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:10.179366 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 721/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 09:13:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:13:10.184594 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9742c00-5165-4f22-99f7-a755d8a7e112 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:10.185074 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9742c00-5165-4f22-99f7-a755d8a7e112 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:10.185182 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9742c00-5165-4f22-99f7-a755d8a7e112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:10.203250 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d9742c00-5165-4f22-99f7-a755d8a7e112 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:13:10.204461 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d9742c00-5165-4f22-99f7-a755d8a7e112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:13:10.213312 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 736/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 09:13:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:13:10.250868 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064e14b1-d50b-4b77-b4d9-05e62f0aa6b3 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:13:10.262203 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064e14b1-d50b-4b77-b4d9-05e62f0aa6b3 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:13:10.265724 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f99a7c-169d-407f-8443-a83f97060271 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:13:10.266360 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f99a7c-169d-407f-8443-a83f97060271 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:10.266648 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f99a7c-169d-407f-8443-a83f97060271 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:10.266928 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f99a7c-169d-407f-8443-a83f97060271 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:10.275883 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f99a7c-169d-407f-8443-a83f97060271 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:13:10.277342 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f99a7c-169d-407f-8443-a83f97060271 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:13:10.287339 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 722/1458] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:13:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:13:12.382229 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf49bbe-28c0-4f99-b205-7020f8d25f49 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:12.382788 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf49bbe-28c0-4f99-b205-7020f8d25f49 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:12.382788 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf49bbe-28c0-4f99-b205-7020f8d25f49 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:12.384177 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 737/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 09:13:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:13:12.389825 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb219730-f282-4d0d-ae41-9de220a09f55 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:12.390280 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb219730-f282-4d0d-ae41-9de220a09f55 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:12.390387 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb219730-f282-4d0d-ae41-9de220a09f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:12.408753 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-fb219730-f282-4d0d-ae41-9de220a09f55 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:13:12.410006 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-fb219730-f282-4d0d-ae41-9de220a09f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:13:12.417964 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 723/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 09:13:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:13:12.455484 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf49bbe-28c0-4f99-b205-7020f8d25f49 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:13:12.467027 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf49bbe-28c0-4f99-b205-7020f8d25f49 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:13:12.470806 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c3eaf5-5648-47c3-b80e-91311f619756 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:13:12.471436 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c3eaf5-5648-47c3-b80e-91311f619756 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:12.471717 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c3eaf5-5648-47c3-b80e-91311f619756 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:12.471990 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c3eaf5-5648-47c3-b80e-91311f619756 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:12.481467 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c3eaf5-5648-47c3-b80e-91311f619756 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:13:12.482877 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c3eaf5-5648-47c3-b80e-91311f619756 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:13:12.493314 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 738/1461] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:13:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:13:14.613725 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f1e2c1-8f4c-45ff-9058-a2f4c45ebbc0 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:14.614311 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f1e2c1-8f4c-45ff-9058-a2f4c45ebbc0 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:14.614311 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f1e2c1-8f4c-45ff-9058-a2f4c45ebbc0 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:14.616341 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 724/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 09:13:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:13:14.621941 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb80a23f-1b43-4386-9508-30a2a690fcb1 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:14.622377 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb80a23f-1b43-4386-9508-30a2a690fcb1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:14.622594 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb80a23f-1b43-4386-9508-30a2a690fcb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:14.628849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=79 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:13:14.641821 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-bb80a23f-1b43-4386-9508-30a2a690fcb1 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:13:14.643020 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bb80a23f-1b43-4386-9508-30a2a690fcb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:13:14.650929 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 739/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 09:13:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:13:14.689384 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f1e2c1-8f4c-45ff-9058-a2f4c45ebbc0 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:13:14.694925 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=73 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:13:14.701277 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f1e2c1-8f4c-45ff-9058-a2f4c45ebbc0 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:13:14.705061 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7455f47a-00fd-434b-9be8-3ea93b440ccc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:13:14.705713 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455f47a-00fd-434b-9be8-3ea93b440ccc service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:14.705998 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455f47a-00fd-434b-9be8-3ea93b440ccc service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:14.706278 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455f47a-00fd-434b-9be8-3ea93b440ccc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:14.715725 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7455f47a-00fd-434b-9be8-3ea93b440ccc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:13:14.717186 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7455f47a-00fd-434b-9be8-3ea93b440ccc service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:13:14.727504 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 725/1464] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:13:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:13:16.795504 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70227efa-9a82-41b1-95fd-49ce3d881d04 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:16.796051 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70227efa-9a82-41b1-95fd-49ce3d881d04 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:16.796051 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70227efa-9a82-41b1-95fd-49ce3d881d04 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:16.797474 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 740/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 09:13:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:13:16.802802 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1949ef1-6c41-4af9-b752-0793f75a134f None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:16.803244 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1949ef1-6c41-4af9-b752-0793f75a134f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:16.803386 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1949ef1-6c41-4af9-b752-0793f75a134f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:16.821077 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-b1949ef1-6c41-4af9-b752-0793f75a134f None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:13:16.822341 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b1949ef1-6c41-4af9-b752-0793f75a134f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:13:16.830084 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 726/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 09:13:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:13:16.866282 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70227efa-9a82-41b1-95fd-49ce3d881d04 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:13:16.876931 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70227efa-9a82-41b1-95fd-49ce3d881d04 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:13:16.880917 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27bbf44-d232-4108-977b-9087f5793e13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:13:16.881650 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27bbf44-d232-4108-977b-9087f5793e13 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:16.881931 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27bbf44-d232-4108-977b-9087f5793e13 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:16.882206 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27bbf44-d232-4108-977b-9087f5793e13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:16.891013 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27bbf44-d232-4108-977b-9087f5793e13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:13:16.892439 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27bbf44-d232-4108-977b-9087f5793e13 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:13:16.902246 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 741/1467] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:13:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:13:18.962889 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef5ae01-a9eb-42e2-9248-34a94d45c14b None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:18.963492 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef5ae01-a9eb-42e2-9248-34a94d45c14b None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:18.963492 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef5ae01-a9eb-42e2-9248-34a94d45c14b None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:18.964802 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 727/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 09:13:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:13:18.970349 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee0a16-bd43-4c32-bcae-36ca6e5c4ead None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:18.970800 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee0a16-bd43-4c32-bcae-36ca6e5c4ead None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:18.970919 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee0a16-bd43-4c32-bcae-36ca6e5c4ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:18.990170 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f8ee0a16-bd43-4c32-bcae-36ca6e5c4ead None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:13:18.991368 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f8ee0a16-bd43-4c32-bcae-36ca6e5c4ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:13:18.999204 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 742/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 09:13:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:13:19.035813 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef5ae01-a9eb-42e2-9248-34a94d45c14b None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:13:19.046637 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef5ae01-a9eb-42e2-9248-34a94d45c14b None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:13:19.050165 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1508ee2d-c6d5-4f51-b9f4-b51e1e0ad217 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:13:19.050802 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1508ee2d-c6d5-4f51-b9f4-b51e1e0ad217 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:19.051089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1508ee2d-c6d5-4f51-b9f4-b51e1e0ad217 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:19.051371 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1508ee2d-c6d5-4f51-b9f4-b51e1e0ad217 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:19.060687 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1508ee2d-c6d5-4f51-b9f4-b51e1e0ad217 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:13:19.062210 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1508ee2d-c6d5-4f51-b9f4-b51e1e0ad217 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:13:19.072397 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 728/1470] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:13:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:13:28.983814 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=11 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:13:29.067661 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=14 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:13:38.999061 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205148d0-cf45-4a02-b06b-2ea90adf6e9e None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:38.999637 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205148d0-cf45-4a02-b06b-2ea90adf6e9e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:38.999637 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205148d0-cf45-4a02-b06b-2ea90adf6e9e None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:39.000973 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 743/1471] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:13:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:13:39.006853 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1985b48-c9d6-4051-a14b-81a8d56dfbb6 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:39.007304 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1985b48-c9d6-4051-a14b-81a8d56dfbb6 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:39.007417 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1985b48-c9d6-4051-a14b-81a8d56dfbb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:39.025031 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-f1985b48-c9d6-4051-a14b-81a8d56dfbb6 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:13:39.026268 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f1985b48-c9d6-4051-a14b-81a8d56dfbb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:13:39.034135 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 729/1472] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:13:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:13:39.118537 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd637fcb-c9cd-445e-a958-058badcde9a7 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:39.118887 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd637fcb-c9cd-445e-a958-058badcde9a7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:39.119395 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd637fcb-c9cd-445e-a958-058badcde9a7 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:39.121025 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 744/1473] 10.222.0.71 () {54 vars in 962 bytes} [Mon Jul 4 09:13:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:13:39.126042 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5458f21-15fe-4a6b-8026-64534f95beb6 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:39.126440 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5458f21-15fe-4a6b-8026-64534f95beb6 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:39.126729 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5458f21-15fe-4a6b-8026-64534f95beb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:39.146326 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-a5458f21-15fe-4a6b-8026-64534f95beb6 None None] MFA Rules not processed for user `192724527d774817bc4581474ac848dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:13:39.147530 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a5458f21-15fe-4a6b-8026-64534f95beb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:13:39.154009 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 730/1474] 10.222.0.71 () {58 vars in 1059 bytes} [Mon Jul 4 09:13:39 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:13:39.161291 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd637fcb-c9cd-445e-a958-058badcde9a7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:13:39.171872 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd637fcb-c9cd-445e-a958-058badcde9a7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:13:39.175486 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eea1690-da6f-48a6-8c30-667be5e9ac0a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '192724527d774817bc4581474ac848dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:13:39.176125 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea1690-da6f-48a6-8c30-667be5e9ac0a service glance] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:39.176411 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea1690-da6f-48a6-8c30-667be5e9ac0a service glance] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:39.176691 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea1690-da6f-48a6-8c30-667be5e9ac0a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:39.185529 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eea1690-da6f-48a6-8c30-667be5e9ac0a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:13:39.186948 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eea1690-da6f-48a6-8c30-667be5e9ac0a service glance] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:13:39.196707 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 745/1475] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:13:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:13:41.294308 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a05a2d-849c-44a5-b7d1-9e3fdaed88f3 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:41.294899 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a05a2d-849c-44a5-b7d1-9e3fdaed88f3 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:41.294899 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a05a2d-849c-44a5-b7d1-9e3fdaed88f3 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:41.296256 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 731/1476] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:13:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:13:41.301514 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d58a7-beeb-4823-b11d-24fcee7272c3 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:41.301951 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d58a7-beeb-4823-b11d-24fcee7272c3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:41.302060 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d58a7-beeb-4823-b11d-24fcee7272c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:41.320487 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-009d58a7-beeb-4823-b11d-24fcee7272c3 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:13:41.321733 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-009d58a7-beeb-4823-b11d-24fcee7272c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:13:41.329541 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 746/1477] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:13:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:13:41.353932 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a05a2d-849c-44a5-b7d1-9e3fdaed88f3 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:13:41.366824 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a05a2d-849c-44a5-b7d1-9e3fdaed88f3 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:13:41.371321 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c46a08-367c-4879-98e1-1dae6b8e4047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:13:41.371967 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c46a08-367c-4879-98e1-1dae6b8e4047 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:41.372254 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c46a08-367c-4879-98e1-1dae6b8e4047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:41.372535 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c46a08-367c-4879-98e1-1dae6b8e4047 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:41.373306 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c46a08-367c-4879-98e1-1dae6b8e4047 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:13:41.375077 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: WARNING py.warnings [None req-17c46a08-367c-4879-98e1-1dae6b8e4047 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 09:13:41.375077 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: warnings.warn(msg) Jul 04 09:13:41.375077 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: Jul 04 09:13:41.376216 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c46a08-367c-4879-98e1-1dae6b8e4047 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:13:41.396895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 732/1478] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 4 09:13:41 2022] GET /identity/v3/projects => generated 2553 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:13:43.462417 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ed31e-2940-47fe-a58d-5e2b592deb27 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:43.462926 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ed31e-2940-47fe-a58d-5e2b592deb27 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:43.463039 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ed31e-2940-47fe-a58d-5e2b592deb27 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:43.464338 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 747/1479] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:13:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:13:43.469840 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcdf325-ae51-4fc7-8f73-515591767f94 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:43.470299 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcdf325-ae51-4fc7-8f73-515591767f94 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:43.470406 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcdf325-ae51-4fc7-8f73-515591767f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:43.489943 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-9bcdf325-ae51-4fc7-8f73-515591767f94 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:13:43.491246 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-9bcdf325-ae51-4fc7-8f73-515591767f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:13:43.499408 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 733/1480] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:13:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:13:43.543578 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48ed31e-2940-47fe-a58d-5e2b592deb27 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:13:43.557869 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48ed31e-2940-47fe-a58d-5e2b592deb27 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:13:43.563208 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4a07a3-5816-4feb-a820-7a33e8ad4617 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15039da209ec445d93d49e2bd6d854eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:13:43.563840 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a07a3-5816-4feb-a820-7a33e8ad4617 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:43.564126 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a07a3-5816-4feb-a820-7a33e8ad4617 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:43.564405 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a07a3-5816-4feb-a820-7a33e8ad4617 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:43.573174 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4a07a3-5816-4feb-a820-7a33e8ad4617 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:13:43.574739 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4a07a3-5816-4feb-a820-7a33e8ad4617 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:13:43.584631 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 748/1481] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:13:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:13:45.812801 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7db52c8-20a1-456c-953e-9c1feb484fd8 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:45.813394 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7db52c8-20a1-456c-953e-9c1feb484fd8 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:45.813394 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7db52c8-20a1-456c-953e-9c1feb484fd8 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:45.814793 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 734/1482] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:13:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:13:45.820542 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f995ab1-fa82-45fc-ac31-4a5c7227aaae None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:45.820999 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f995ab1-fa82-45fc-ac31-4a5c7227aaae None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:45.821126 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f995ab1-fa82-45fc-ac31-4a5c7227aaae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:45.838523 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4f995ab1-fa82-45fc-ac31-4a5c7227aaae None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:13:45.839725 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4f995ab1-fa82-45fc-ac31-4a5c7227aaae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:13:45.848278 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 749/1483] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:13:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:13:45.884286 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7db52c8-20a1-456c-953e-9c1feb484fd8 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:13:45.895128 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7db52c8-20a1-456c-953e-9c1feb484fd8 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:13:45.898696 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1520c11-d1a4-40f3-8abb-c8806042817b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:13:45.899330 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1520c11-d1a4-40f3-8abb-c8806042817b service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:45.899617 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1520c11-d1a4-40f3-8abb-c8806042817b service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:45.899896 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1520c11-d1a4-40f3-8abb-c8806042817b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:45.908775 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1520c11-d1a4-40f3-8abb-c8806042817b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:13:45.910852 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1520c11-d1a4-40f3-8abb-c8806042817b service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:13:45.920204 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 735/1484] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:13:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:13:48.054872 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2473af1e-7da8-4961-87dc-6b7138469054 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:48.055488 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2473af1e-7da8-4961-87dc-6b7138469054 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:48.055488 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2473af1e-7da8-4961-87dc-6b7138469054 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:48.056797 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 750/1485] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:13:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:13:48.062113 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3f9fc-555b-4f76-893e-eef6d8d708a7 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:48.062577 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3f9fc-555b-4f76-893e-eef6d8d708a7 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:48.062702 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3f9fc-555b-4f76-893e-eef6d8d708a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:48.082206 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-84b3f9fc-555b-4f76-893e-eef6d8d708a7 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:13:48.083477 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-84b3f9fc-555b-4f76-893e-eef6d8d708a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:13:48.093849 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 736/1486] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:13:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:13:48.132209 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2473af1e-7da8-4961-87dc-6b7138469054 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:13:48.142960 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2473af1e-7da8-4961-87dc-6b7138469054 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:13:48.147317 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883f2935-e796-4fe0-b8f0-e66cfb0827ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:13:48.148010 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f2935-e796-4fe0-b8f0-e66cfb0827ed service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:48.148298 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f2935-e796-4fe0-b8f0-e66cfb0827ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:48.148580 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f2935-e796-4fe0-b8f0-e66cfb0827ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:48.158624 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883f2935-e796-4fe0-b8f0-e66cfb0827ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:13:48.160585 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883f2935-e796-4fe0-b8f0-e66cfb0827ed service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:13:48.170345 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 751/1487] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:13:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:13:50.364349 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f607b-8404-40e9-a81b-ba377c31d662 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:50.364905 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f607b-8404-40e9-a81b-ba377c31d662 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:50.364905 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f607b-8404-40e9-a81b-ba377c31d662 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:50.366315 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 737/1488] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:13:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:13:50.371398 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfee9217-2549-4199-a6f7-4d69bbda94a4 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:50.371824 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfee9217-2549-4199-a6f7-4d69bbda94a4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:50.371993 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfee9217-2549-4199-a6f7-4d69bbda94a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:50.389917 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-cfee9217-2549-4199-a6f7-4d69bbda94a4 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:13:50.391111 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cfee9217-2549-4199-a6f7-4d69bbda94a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:13:50.399214 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 752/1489] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:13:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:13:50.441744 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023f607b-8404-40e9-a81b-ba377c31d662 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:13:50.443417 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-023f607b-8404-40e9-a81b-ba377c31d662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:13:50.456637 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023f607b-8404-40e9-a81b-ba377c31d662 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:13:50.460158 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896e6fe4-c6fc-4219-8687-ac77678dd0be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:13:50.460794 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e6fe4-c6fc-4219-8687-ac77678dd0be service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:50.461112 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e6fe4-c6fc-4219-8687-ac77678dd0be service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:50.461393 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e6fe4-c6fc-4219-8687-ac77678dd0be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:50.470456 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896e6fe4-c6fc-4219-8687-ac77678dd0be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:13:50.472909 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896e6fe4-c6fc-4219-8687-ac77678dd0be service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:13:50.482186 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 738/1490] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:13:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:13:52.707514 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbac244-cdd1-450b-8252-fc55719de882 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:52.708110 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbac244-cdd1-450b-8252-fc55719de882 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:52.708110 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbac244-cdd1-450b-8252-fc55719de882 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:52.709483 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 753/1491] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 4 09:13:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:13:52.714718 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcba3ea1-78ef-46e6-aefb-2eaee9eb9b03 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:52.715154 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcba3ea1-78ef-46e6-aefb-2eaee9eb9b03 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:52.715276 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcba3ea1-78ef-46e6-aefb-2eaee9eb9b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:52.733312 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-fcba3ea1-78ef-46e6-aefb-2eaee9eb9b03 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:13:52.734498 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-fcba3ea1-78ef-46e6-aefb-2eaee9eb9b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:13:52.742731 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 739/1492] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 4 09:13:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:13:52.781936 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbac244-cdd1-450b-8252-fc55719de882 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:13:52.793259 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbac244-cdd1-450b-8252-fc55719de882 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:13:52.796825 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c79cb3b-5b73-4205-9636-37e5ab506c14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:13:52.797497 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79cb3b-5b73-4205-9636-37e5ab506c14 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:52.797811 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79cb3b-5b73-4205-9636-37e5ab506c14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:52.798091 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79cb3b-5b73-4205-9636-37e5ab506c14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:52.807112 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c79cb3b-5b73-4205-9636-37e5ab506c14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:13:52.808534 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c79cb3b-5b73-4205-9636-37e5ab506c14 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:13:52.818442 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 754/1493] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:13:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:14:02.728340 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=33 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:14:02.813739 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=33 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:19.300771 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d8ce0-b2ba-4ae8-af1e-74fffd1c996d None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:19.301401 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d8ce0-b2ba-4ae8-af1e-74fffd1c996d None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:19.301401 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d8ce0-b2ba-4ae8-af1e-74fffd1c996d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:19.330232 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-197d8ce0-b2ba-4ae8-af1e-74fffd1c996d None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:19.331429 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-197d8ce0-b2ba-4ae8-af1e-74fffd1c996d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:19.339439 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 740/1494] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:15:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:19.348908 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a82486-216d-4f9a-aa9f-8678b2461fcd None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:19.349441 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a82486-216d-4f9a-aa9f-8678b2461fcd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:19.349545 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a82486-216d-4f9a-aa9f-8678b2461fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:19.367447 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b1a82486-216d-4f9a-aa9f-8678b2461fcd None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:19.368631 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b1a82486-216d-4f9a-aa9f-8678b2461fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:19.376819 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 755/1495] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:15:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:19.381528 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197d8ce0-b2ba-4ae8-af1e-74fffd1c996d None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:19.393739 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197d8ce0-b2ba-4ae8-af1e-74fffd1c996d None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:19.398473 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23684a6c-73d6-4614-923c-5d46333e07db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:19.399133 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23684a6c-73d6-4614-923c-5d46333e07db admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:19.399417 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23684a6c-73d6-4614-923c-5d46333e07db admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:19.399693 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23684a6c-73d6-4614-923c-5d46333e07db admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:19.400443 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23684a6c-73d6-4614-923c-5d46333e07db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:15:19.400751 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23684a6c-73d6-4614-923c-5d46333e07db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:19.401800 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: WARNING py.warnings [None req-23684a6c-73d6-4614-923c-5d46333e07db admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 04 09:15:19.401800 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: warnings.warn(msg) Jul 04 09:15:19.401800 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: Jul 04 09:15:19.402676 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23684a6c-73d6-4614-923c-5d46333e07db admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:19.415132 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 741/1496] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:15:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:15:19.423906 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a82486-216d-4f9a-aa9f-8678b2461fcd None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:19.433430 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a82486-216d-4f9a-aa9f-8678b2461fcd None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:19.438122 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa3dc20-c364-40e6-b212-71895e48d1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:19.438771 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa3dc20-c364-40e6-b212-71895e48d1ca admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:19.439060 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa3dc20-c364-40e6-b212-71895e48d1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:19.439340 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa3dc20-c364-40e6-b212-71895e48d1ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:19.440047 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa3dc20-c364-40e6-b212-71895e48d1ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:19.441849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-cfa3dc20-c364-40e6-b212-71895e48d1ca admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 09:15:19.441849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Jul 04 09:15:19.441849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: Jul 04 09:15:19.443047 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa3dc20-c364-40e6-b212-71895e48d1ca admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:19.480838 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 756/1497] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:15:19 2022] POST /identity/v3/projects => generated 375 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:19.485422 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23684a6c-73d6-4614-923c-5d46333e07db admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:19.495278 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23684a6c-73d6-4614-923c-5d46333e07db admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:19.550901 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb9af57-1d49-4da7-b394-37c5a587e85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:19.551493 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb9af57-1d49-4da7-b394-37c5a587e85b admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:19.551781 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb9af57-1d49-4da7-b394-37c5a587e85b admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:19.552059 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb9af57-1d49-4da7-b394-37c5a587e85b admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:19.552756 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb9af57-1d49-4da7-b394-37c5a587e85b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:19.553848 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: WARNING py.warnings [None req-fcb9af57-1d49-4da7-b394-37c5a587e85b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 09:15:19.553848 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: warnings.warn(msg) Jul 04 09:15:19.553848 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: Jul 04 09:15:19.555046 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb9af57-1d49-4da7-b394-37c5a587e85b admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:19.574717 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 742/1498] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:15:19 2022] POST /identity/v3/users => generated 288 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:19.579682 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa3dc20-c364-40e6-b212-71895e48d1ca admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:19.589685 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa3dc20-c364-40e6-b212-71895e48d1ca admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:19.594454 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915f639d-c632-468f-9554-a6f036c7ff18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:19.595111 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f639d-c632-468f-9554-a6f036c7ff18 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:19.595397 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f639d-c632-468f-9554-a6f036c7ff18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:19.595675 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f639d-c632-468f-9554-a6f036c7ff18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:19.596392 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915f639d-c632-468f-9554-a6f036c7ff18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:19.597429 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-915f639d-c632-468f-9554-a6f036c7ff18 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 04 09:15:19.597429 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Jul 04 09:15:19.597429 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: Jul 04 09:15:19.598299 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915f639d-c632-468f-9554-a6f036c7ff18 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:19.611571 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 757/1499] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:19.615584 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb9af57-1d49-4da7-b394-37c5a587e85b admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:19.626444 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb9af57-1d49-4da7-b394-37c5a587e85b admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:19.631328 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc3a835-a9c3-4b32-8a9b-d0a2425daef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:19.632002 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3a835-a9c3-4b32-8a9b-d0a2425daef8 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:19.632287 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3a835-a9c3-4b32-8a9b-d0a2425daef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:19.632562 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3a835-a9c3-4b32-8a9b-d0a2425daef8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:19.633337 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc3a835-a9c3-4b32-8a9b-d0a2425daef8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:19.634832 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: WARNING py.warnings [None req-5dc3a835-a9c3-4b32-8a9b-d0a2425daef8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 04 09:15:19.634832 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: warnings.warn(msg) Jul 04 09:15:19.634832 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: Jul 04 09:15:19.635718 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc3a835-a9c3-4b32-8a9b-d0a2425daef8 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:19.650637 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 743/1500] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:19.654820 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915f639d-c632-468f-9554-a6f036c7ff18 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:19.664541 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915f639d-c632-468f-9554-a6f036c7ff18 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:19.669110 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b118df8e-2b95-4062-b15f-ea3baffc5151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:19.670023 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b118df8e-2b95-4062-b15f-ea3baffc5151 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:19.670350 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b118df8e-2b95-4062-b15f-ea3baffc5151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:19.670630 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b118df8e-2b95-4062-b15f-ea3baffc5151 admin admin] PATH_INFO: `/v3/projects/281cc005a9ac4fdd83646de32901340e/users/f3b87daeedac499893748c24768dedaa/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:19.685958 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b118df8e-2b95-4062-b15f-ea3baffc5151 admin admin] RBAC: Authorizing `identity:create_grant(project_id=281cc005a9ac4fdd83646de32901340e, user_id=f3b87daeedac499893748c24768dedaa, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:19.687177 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-b118df8e-2b95-4062-b15f-ea3baffc5151 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 09:15:19.687177 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Jul 04 09:15:19.687177 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: Jul 04 09:15:19.689138 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b118df8e-2b95-4062-b15f-ea3baffc5151 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:19.702429 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 758/1501] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:15:19 2022] PUT /identity/v3/projects/281cc005a9ac4fdd83646de32901340e/users/f3b87daeedac499893748c24768dedaa/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:15:19.714581 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5faf9a-dab6-4a75-ba4d-2261574e462d None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:19.715133 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5faf9a-dab6-4a75-ba4d-2261574e462d None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:19.715427 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5faf9a-dab6-4a75-ba4d-2261574e462d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:19.741206 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-7e5faf9a-dab6-4a75-ba4d-2261574e462d None None] MFA Rules not processed for user `f3b87daeedac499893748c24768dedaa`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:19.742430 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7e5faf9a-dab6-4a75-ba4d-2261574e462d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:19.805802 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 744/1502] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:15:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:19.819223 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc10a5e-27bd-43f1-b4ad-070a60f644d9 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:19.819515 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc10a5e-27bd-43f1-b4ad-070a60f644d9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:19.819796 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc10a5e-27bd-43f1-b4ad-070a60f644d9 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:19.821245 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 759/1503] 10.222.0.71 () {54 vars in 963 bytes} [Mon Jul 4 09:15:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:15:19.826609 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d39d7fe-ab8e-4595-b323-d8b5cd403389 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:19.826912 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d39d7fe-ab8e-4595-b323-d8b5cd403389 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:19.827196 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d39d7fe-ab8e-4595-b323-d8b5cd403389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:19.844888 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-9d39d7fe-ab8e-4595-b323-d8b5cd403389 None None] MFA Rules not processed for user `f6e50a3715a14714b3951d36338dde42`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:19.846124 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-9d39d7fe-ab8e-4595-b323-d8b5cd403389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:19.889108 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:19.897646 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 745/1504] 10.222.0.71 () {58 vars in 1060 bytes} [Mon Jul 4 09:15:19 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:19.905242 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc10a5e-27bd-43f1-b4ad-070a60f644d9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:19.915936 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc10a5e-27bd-43f1-b4ad-070a60f644d9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:19.919455 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a952d0d4-3f57-478e-801a-3ccdeb4234a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:19.920089 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a952d0d4-3f57-478e-801a-3ccdeb4234a6 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:19.920373 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a952d0d4-3f57-478e-801a-3ccdeb4234a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:19.920651 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a952d0d4-3f57-478e-801a-3ccdeb4234a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:19.928787 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a952d0d4-3f57-478e-801a-3ccdeb4234a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:19.930239 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a952d0d4-3f57-478e-801a-3ccdeb4234a6 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:19.939413 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 760/1505] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:19 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:15:19.981188 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e1ff0-d55b-4b87-9da4-074993391f54 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:19.981481 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e1ff0-d55b-4b87-9da4-074993391f54 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:19.981789 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e1ff0-d55b-4b87-9da4-074993391f54 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:19.983628 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 746/1506] 10.222.0.71 () {52 vars in 829 bytes} [Mon Jul 4 09:15:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:15:19.987449 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a952d0d4-3f57-478e-801a-3ccdeb4234a6 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:20.048175 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a952d0d4-3f57-478e-801a-3ccdeb4234a6 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:20.052693 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0075a2b-d1e4-4c09-aefe-63fa7ad3c560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:20.053586 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0075a2b-d1e4-4c09-aefe-63fa7ad3c560 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:20.053871 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0075a2b-d1e4-4c09-aefe-63fa7ad3c560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:20.054151 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0075a2b-d1e4-4c09-aefe-63fa7ad3c560 admin admin] PATH_INFO: `/v3/users/f3b87daeedac499893748c24768dedaa` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:20.055351 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0075a2b-d1e4-4c09-aefe-63fa7ad3c560 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3b87daeedac499893748c24768dedaa)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:20.056415 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-a0075a2b-d1e4-4c09-aefe-63fa7ad3c560 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 09:15:20.056415 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Jul 04 09:15:20.056415 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: Jul 04 09:15:20.057607 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0075a2b-d1e4-4c09-aefe-63fa7ad3c560 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:20.121517 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a0075a2b-d1e4-4c09-aefe-63fa7ad3c560 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3b87daeedac499893748c24768dedaa'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:20.122256 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a0075a2b-d1e4-4c09-aefe-63fa7ad3c560 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3b87daeedac499893748c24768dedaa'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:20.133335 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a0075a2b-d1e4-4c09-aefe-63fa7ad3c560 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3b87daeedac499893748c24768dedaa'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:20.151970 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a0075a2b-d1e4-4c09-aefe-63fa7ad3c560 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3b87daeedac499893748c24768dedaa'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:20.152658 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a0075a2b-d1e4-4c09-aefe-63fa7ad3c560 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3b87daeedac499893748c24768dedaa'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:20.161404 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:20.173835 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 761/1507] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:15:19 2022] DELETE /identity/v3/users/f3b87daeedac499893748c24768dedaa => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:20.178476 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d145c951-bd59-40b3-90b7-5c5cc6a1647b None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:20.178768 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d145c951-bd59-40b3-90b7-5c5cc6a1647b None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:20.179073 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d145c951-bd59-40b3-90b7-5c5cc6a1647b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:20.198110 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-d145c951-bd59-40b3-90b7-5c5cc6a1647b None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:20.199332 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d145c951-bd59-40b3-90b7-5c5cc6a1647b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:20.257458 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 747/1508] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:15:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:20.266913 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0075a2b-d1e4-4c09-aefe-63fa7ad3c560 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:20.268382 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a0075a2b-d1e4-4c09-aefe-63fa7ad3c560 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:20.329459 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0075a2b-d1e4-4c09-aefe-63fa7ad3c560 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:20.332927 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57209cc4-d222-45b9-8576-9bf462ef38ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:20.333717 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57209cc4-d222-45b9-8576-9bf462ef38ad service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:20.333998 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57209cc4-d222-45b9-8576-9bf462ef38ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:20.334273 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57209cc4-d222-45b9-8576-9bf462ef38ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:20.344282 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57209cc4-d222-45b9-8576-9bf462ef38ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:20.345742 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57209cc4-d222-45b9-8576-9bf462ef38ad service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:20.354942 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 762/1509] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:15:20.679075 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d145c951-bd59-40b3-90b7-5c5cc6a1647b None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:20.680522 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d145c951-bd59-40b3-90b7-5c5cc6a1647b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:20.692650 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d145c951-bd59-40b3-90b7-5c5cc6a1647b None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:20.697246 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6394b00-0120-4e8d-b83b-242ab9146f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:20.698117 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6394b00-0120-4e8d-b83b-242ab9146f2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:20.698396 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6394b00-0120-4e8d-b83b-242ab9146f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:20.698670 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6394b00-0120-4e8d-b83b-242ab9146f2b admin admin] PATH_INFO: `/v3/projects/281cc005a9ac4fdd83646de32901340e` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:20.699867 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6394b00-0120-4e8d-b83b-242ab9146f2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=281cc005a9ac4fdd83646de32901340e)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:20.701620 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: WARNING py.warnings [None req-b6394b00-0120-4e8d-b83b-242ab9146f2b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 09:15:20.701620 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: warnings.warn(msg) Jul 04 09:15:20.701620 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: Jul 04 09:15:20.702781 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6394b00-0120-4e8d-b83b-242ab9146f2b admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:20.761917 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b6394b00-0120-4e8d-b83b-242ab9146f2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '281cc005a9ac4fdd83646de32901340e', 'request_id': 'req-b6394b00-0120-4e8d-b83b-242ab9146f2b'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:20.768301 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b6394b00-0120-4e8d-b83b-242ab9146f2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '281cc005a9ac4fdd83646de32901340e', 'request_id': 'req-b6394b00-0120-4e8d-b83b-242ab9146f2b'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:20.781941 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b6394b00-0120-4e8d-b83b-242ab9146f2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '281cc005a9ac4fdd83646de32901340e', 'request_id': 'req-b6394b00-0120-4e8d-b83b-242ab9146f2b'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:20.795764 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b6394b00-0120-4e8d-b83b-242ab9146f2b admin admin] The token cache is being invalidate because project 281cc005a9ac4fdd83646de32901340e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:15:20.796051 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b6394b00-0120-4e8d-b83b-242ab9146f2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:20.796774 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b6394b00-0120-4e8d-b83b-242ab9146f2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:20.798340 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 748/1510] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:15:20 2022] DELETE /identity/v3/projects/281cc005a9ac4fdd83646de32901340e => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:30.350822 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=29 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:30.792575 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:17:02.130200 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a4830b-ecaa-4bfe-99b4-443838b38160 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:02.130824 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a4830b-ecaa-4bfe-99b4-443838b38160 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:02.130824 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a4830b-ecaa-4bfe-99b4-443838b38160 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:02.132135 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 763/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 09:17:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:17:02.137474 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b543b51-26aa-4c25-90f8-502577040473 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:02.138019 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b543b51-26aa-4c25-90f8-502577040473 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:02.138126 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b543b51-26aa-4c25-90f8-502577040473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:02.156022 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-1b543b51-26aa-4c25-90f8-502577040473 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:17:02.157275 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-1b543b51-26aa-4c25-90f8-502577040473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:17:02.215436 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 749/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 09:17:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:17:02.384199 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a4830b-ecaa-4bfe-99b4-443838b38160 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:17:02.386000 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d8a4830b-ecaa-4bfe-99b4-443838b38160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:17:02.459548 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a4830b-ecaa-4bfe-99b4-443838b38160 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:17:02.466188 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20218f3c-2d0a-4ad4-b172-f3a16eeee20f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15039da209ec445d93d49e2bd6d854eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:17:02.467081 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20218f3c-2d0a-4ad4-b172-f3a16eeee20f service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:02.467368 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20218f3c-2d0a-4ad4-b172-f3a16eeee20f service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:02.467641 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20218f3c-2d0a-4ad4-b172-f3a16eeee20f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:02.475728 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20218f3c-2d0a-4ad4-b172-f3a16eeee20f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:17:02.477339 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20218f3c-2d0a-4ad4-b172-f3a16eeee20f service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:17:02.488016 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 764/1513] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:17:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:17:03.336341 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d7564-2b4d-4f75-9e29-fbcfad8c851f None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:03.336922 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d7564-2b4d-4f75-9e29-fbcfad8c851f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:03.336922 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d7564-2b4d-4f75-9e29-fbcfad8c851f None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:03.338339 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 750/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 09:17:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:17:03.343488 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db74dfd-07ec-4968-adb7-82f35ba5427b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:03.343903 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db74dfd-07ec-4968-adb7-82f35ba5427b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:03.344079 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db74dfd-07ec-4968-adb7-82f35ba5427b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:03.362961 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9db74dfd-07ec-4968-adb7-82f35ba5427b None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:17:03.364159 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9db74dfd-07ec-4968-adb7-82f35ba5427b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:17:03.374074 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 765/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 09:17:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:17:03.529600 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d7564-2b4d-4f75-9e29-fbcfad8c851f None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:17:03.544755 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d7564-2b4d-4f75-9e29-fbcfad8c851f None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:17:03.550253 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd289de0-e62c-4340-a28a-962c0afb875b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15039da209ec445d93d49e2bd6d854eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:17:03.550896 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd289de0-e62c-4340-a28a-962c0afb875b service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:03.551188 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd289de0-e62c-4340-a28a-962c0afb875b service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:03.551470 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd289de0-e62c-4340-a28a-962c0afb875b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:03.560501 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd289de0-e62c-4340-a28a-962c0afb875b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:17:03.562185 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd289de0-e62c-4340-a28a-962c0afb875b service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:17:03.571681 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 751/1516] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:17:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:17:04.315868 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6336631b-cd5f-4d53-9f8e-409219a17b92 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:04.316158 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6336631b-cd5f-4d53-9f8e-409219a17b92 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:04.316438 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6336631b-cd5f-4d53-9f8e-409219a17b92 None None] PATH_INFO: `/v3` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:04.317839 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 766/1517] 10.222.0.65 () {54 vars in 925 bytes} [Mon Jul 4 09:17:04 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:17:04.324410 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652522cd-0c3f-42dd-9a71-6533909aa26c None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:04.324707 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652522cd-0c3f-42dd-9a71-6533909aa26c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:04.324993 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652522cd-0c3f-42dd-9a71-6533909aa26c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:04.343242 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-652522cd-0c3f-42dd-9a71-6533909aa26c None None] MFA Rules not processed for user `293c695f15c8437d81a619f840318f44`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:17:04.344476 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-652522cd-0c3f-42dd-9a71-6533909aa26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:17:04.410982 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 752/1518] 10.222.0.65 () {58 vars in 1015 bytes} [Mon Jul 4 09:17:04 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:17:08.284558 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bbe468-e644-4bfa-9544-4b76be11c407 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:08.285190 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bbe468-e644-4bfa-9544-4b76be11c407 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:08.285190 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bbe468-e644-4bfa-9544-4b76be11c407 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:08.286408 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 767/1519] 10.222.0.71 () {54 vars in 960 bytes} [Mon Jul 4 09:17:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:17:08.291414 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd2935-c64e-4226-bd84-79795e13b5d3 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:08.292565 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd2935-c64e-4226-bd84-79795e13b5d3 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:08.292565 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd2935-c64e-4226-bd84-79795e13b5d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:08.310402 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-64fd2935-c64e-4226-bd84-79795e13b5d3 None None] MFA Rules not processed for user `293c695f15c8437d81a619f840318f44`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:17:08.311614 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-64fd2935-c64e-4226-bd84-79795e13b5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:17:08.321094 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 753/1520] 10.222.0.71 () {58 vars in 1057 bytes} [Mon Jul 4 09:17:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:17:08.329248 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bbe468-e644-4bfa-9544-4b76be11c407 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:17:08.343502 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bbe468-e644-4bfa-9544-4b76be11c407 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:17:08.348444 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe2730-0fe1-4002-95a8-67440b77b44d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '293c695f15c8437d81a619f840318f44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:17:08.349116 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe2730-0fe1-4002-95a8-67440b77b44d service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:08.349400 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe2730-0fe1-4002-95a8-67440b77b44d service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:08.349675 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe2730-0fe1-4002-95a8-67440b77b44d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:08.358525 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbe2730-0fe1-4002-95a8-67440b77b44d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:17:08.360125 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbe2730-0fe1-4002-95a8-67440b77b44d service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:17:08.369619 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 768/1521] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:17:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:17:14.813896 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ebbd61-d325-4c1c-af78-7aaef4a34023 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:14.814475 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ebbd61-d325-4c1c-af78-7aaef4a34023 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:14.814475 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ebbd61-d325-4c1c-af78-7aaef4a34023 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:14.815829 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 754/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 09:17:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:17:14.820662 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0a74b-eccb-40cf-9e8c-b227b7049a8c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:14.821086 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0a74b-eccb-40cf-9e8c-b227b7049a8c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:14.821283 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0a74b-eccb-40cf-9e8c-b227b7049a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:14.840385 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-93d0a74b-eccb-40cf-9e8c-b227b7049a8c None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:17:14.841623 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-93d0a74b-eccb-40cf-9e8c-b227b7049a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:17:14.849301 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 769/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 09:17:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:17:15.006046 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ebbd61-d325-4c1c-af78-7aaef4a34023 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:17:15.020300 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ebbd61-d325-4c1c-af78-7aaef4a34023 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:17:15.025778 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab05afc-1c27-4d38-bf7a-20f0e5228fd0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15039da209ec445d93d49e2bd6d854eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:17:15.027452 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab05afc-1c27-4d38-bf7a-20f0e5228fd0 service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:15.027746 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab05afc-1c27-4d38-bf7a-20f0e5228fd0 service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:15.028028 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab05afc-1c27-4d38-bf7a-20f0e5228fd0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:15.036143 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab05afc-1c27-4d38-bf7a-20f0e5228fd0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:17:15.037782 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab05afc-1c27-4d38-bf7a-20f0e5228fd0 service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:17:15.047982 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 755/1524] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:17:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:17:15.869427 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e1949f-5592-4474-baaf-c25a964644e9 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:15.870425 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e1949f-5592-4474-baaf-c25a964644e9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:15.870425 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e1949f-5592-4474-baaf-c25a964644e9 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:15.871382 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 770/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 09:17:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:17:15.876141 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce6ac4-7e9a-4e4b-b893-5065928de236 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:15.876583 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce6ac4-7e9a-4e4b-b893-5065928de236 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:15.876706 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce6ac4-7e9a-4e4b-b893-5065928de236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:15.894312 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-c9ce6ac4-7e9a-4e4b-b893-5065928de236 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:17:15.895508 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c9ce6ac4-7e9a-4e4b-b893-5065928de236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:17:15.903871 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 756/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 09:17:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:17:16.056669 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e1949f-5592-4474-baaf-c25a964644e9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:17:16.071309 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e1949f-5592-4474-baaf-c25a964644e9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:17:16.077023 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3dc30-47e1-4ffd-9bf6-121a457b9cc1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15039da209ec445d93d49e2bd6d854eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:17:16.077680 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3dc30-47e1-4ffd-9bf6-121a457b9cc1 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:16.077962 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3dc30-47e1-4ffd-9bf6-121a457b9cc1 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:16.078238 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3dc30-47e1-4ffd-9bf6-121a457b9cc1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:16.086246 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3dc30-47e1-4ffd-9bf6-121a457b9cc1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:17:16.087812 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3dc30-47e1-4ffd-9bf6-121a457b9cc1 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:17:16.098125 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 771/1527] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:17:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:17:20.115857 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53c67a-503a-4c53-874d-19eef182b48b None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:20.116405 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53c67a-503a-4c53-874d-19eef182b48b None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:20.116405 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53c67a-503a-4c53-874d-19eef182b48b None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:20.117826 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 757/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 09:17:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:17:20.123134 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ebc76-316d-4208-8344-423f110a2081 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:20.123550 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ebc76-316d-4208-8344-423f110a2081 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:20.123697 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ebc76-316d-4208-8344-423f110a2081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:20.142102 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e57ebc76-316d-4208-8344-423f110a2081 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:17:20.143300 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e57ebc76-316d-4208-8344-423f110a2081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:17:20.152005 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 772/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 09:17:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:17:20.223623 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff53c67a-503a-4c53-874d-19eef182b48b None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:17:20.225406 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ff53c67a-503a-4c53-874d-19eef182b48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:17:20.283613 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff53c67a-503a-4c53-874d-19eef182b48b None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:17:20.287173 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ee7a04-1689-4229-86ed-be9b673bbe5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:17:20.287850 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ee7a04-1689-4229-86ed-be9b673bbe5b service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:20.288138 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ee7a04-1689-4229-86ed-be9b673bbe5b service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:20.288421 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ee7a04-1689-4229-86ed-be9b673bbe5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:20.297840 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ee7a04-1689-4229-86ed-be9b673bbe5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:17:20.299281 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ee7a04-1689-4229-86ed-be9b673bbe5b service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:17:20.304149 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:17:20.310271 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 758/1530] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:17:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:17:22.335761 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6420edd-d344-47d4-9812-7eee119ffe4f None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:22.336290 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6420edd-d344-47d4-9812-7eee119ffe4f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:22.336396 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6420edd-d344-47d4-9812-7eee119ffe4f None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:22.337768 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 773/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 09:17:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:17:22.343999 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d8272f-29db-46ad-afe8-53b7332c20eb None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:22.344448 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d8272f-29db-46ad-afe8-53b7332c20eb None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:22.344575 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d8272f-29db-46ad-afe8-53b7332c20eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:22.363924 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-e5d8272f-29db-46ad-afe8-53b7332c20eb None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:17:22.365165 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e5d8272f-29db-46ad-afe8-53b7332c20eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:17:22.373008 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 759/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 09:17:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:17:22.409586 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6420edd-d344-47d4-9812-7eee119ffe4f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:17:22.420603 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6420edd-d344-47d4-9812-7eee119ffe4f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:17:22.424177 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86b29d1-feb7-4c02-b540-a7fa5cddb608 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:17:22.424811 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b29d1-feb7-4c02-b540-a7fa5cddb608 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:22.425133 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b29d1-feb7-4c02-b540-a7fa5cddb608 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:22.425417 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b29d1-feb7-4c02-b540-a7fa5cddb608 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:22.434929 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86b29d1-feb7-4c02-b540-a7fa5cddb608 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:17:22.437367 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86b29d1-feb7-4c02-b540-a7fa5cddb608 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:17:22.452053 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 774/1533] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:17:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:17:26.060660 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88560718-eaa7-4e79-bac7-d602b9e93c1b None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:26.061265 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88560718-eaa7-4e79-bac7-d602b9e93c1b None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:26.061265 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88560718-eaa7-4e79-bac7-d602b9e93c1b None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:26.062655 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 760/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 09:17:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:17:26.067899 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd1816e-487f-41fd-8a45-763031cca80c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:26.068348 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd1816e-487f-41fd-8a45-763031cca80c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:26.068456 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd1816e-487f-41fd-8a45-763031cca80c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:26.086904 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ecd1816e-487f-41fd-8a45-763031cca80c None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:17:26.088992 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ecd1816e-487f-41fd-8a45-763031cca80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:17:26.096846 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 775/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 09:17:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:17:26.397692 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88560718-eaa7-4e79-bac7-d602b9e93c1b None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:17:26.401401 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-88560718-eaa7-4e79-bac7-d602b9e93c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:17:26.463813 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88560718-eaa7-4e79-bac7-d602b9e93c1b None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:17:26.467410 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265009fe-897c-4588-ac96-7287a1e47b6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:17:26.468061 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265009fe-897c-4588-ac96-7287a1e47b6c service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:26.468350 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265009fe-897c-4588-ac96-7287a1e47b6c service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:26.468639 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265009fe-897c-4588-ac96-7287a1e47b6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:26.477561 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265009fe-897c-4588-ac96-7287a1e47b6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:17:26.479166 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265009fe-897c-4588-ac96-7287a1e47b6c service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:17:26.488559 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 761/1536] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:17:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:17:28.562278 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be00149-1557-4c1f-9b9e-aa17ad1ef742 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:28.562814 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be00149-1557-4c1f-9b9e-aa17ad1ef742 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:28.562814 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be00149-1557-4c1f-9b9e-aa17ad1ef742 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:28.564138 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 776/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 09:17:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:17:28.569263 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18758919-16fe-4daa-b561-03afc251ead1 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:28.569712 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18758919-16fe-4daa-b561-03afc251ead1 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:28.569826 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18758919-16fe-4daa-b561-03afc251ead1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:28.587711 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-18758919-16fe-4daa-b561-03afc251ead1 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:17:28.589699 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-18758919-16fe-4daa-b561-03afc251ead1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:17:28.597505 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 762/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 09:17:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:17:28.896784 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be00149-1557-4c1f-9b9e-aa17ad1ef742 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:17:28.898727 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2be00149-1557-4c1f-9b9e-aa17ad1ef742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:17:28.912480 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be00149-1557-4c1f-9b9e-aa17ad1ef742 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:17:28.915894 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28026813-cb7c-4776-97ce-1e4948174793 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:17:28.917081 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28026813-cb7c-4776-97ce-1e4948174793 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:28.917368 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28026813-cb7c-4776-97ce-1e4948174793 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:28.917642 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28026813-cb7c-4776-97ce-1e4948174793 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:28.926318 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28026813-cb7c-4776-97ce-1e4948174793 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:17:28.927722 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28026813-cb7c-4776-97ce-1e4948174793 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:17:28.937373 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 777/1539] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:17:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:17:38.582549 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=32 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:17:38.933340 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=70 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:18:57.436840 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab04b36-b46d-4302-be9b-2ce443b0c273 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:57.437517 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab04b36-b46d-4302-be9b-2ce443b0c273 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:57.437517 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab04b36-b46d-4302-be9b-2ce443b0c273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:57.464680 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-cab04b36-b46d-4302-be9b-2ce443b0c273 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:18:57.467730 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-cab04b36-b46d-4302-be9b-2ce443b0c273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:18:57.479058 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 763/1540] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:18:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:18:57.488477 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3324aaa-3c9e-4709-aadc-cc8b8f968ac5 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:57.490062 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3324aaa-3c9e-4709-aadc-cc8b8f968ac5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:57.490550 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3324aaa-3c9e-4709-aadc-cc8b8f968ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:57.518194 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d3324aaa-3c9e-4709-aadc-cc8b8f968ac5 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:18:57.519545 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d3324aaa-3c9e-4709-aadc-cc8b8f968ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:18:57.531430 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 778/1541] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:18:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:18:57.536448 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab04b36-b46d-4302-be9b-2ce443b0c273 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:57.548184 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab04b36-b46d-4302-be9b-2ce443b0c273 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:57.553267 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc98c330-5397-41d1-9c4f-6f7bf6033f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:57.554174 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc98c330-5397-41d1-9c4f-6f7bf6033f59 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:57.554664 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc98c330-5397-41d1-9c4f-6f7bf6033f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:57.555152 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc98c330-5397-41d1-9c4f-6f7bf6033f59 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:57.556413 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc98c330-5397-41d1-9c4f-6f7bf6033f59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:18:57.556945 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc98c330-5397-41d1-9c4f-6f7bf6033f59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:57.558833 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc98c330-5397-41d1-9c4f-6f7bf6033f59 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:57.572942 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 764/1542] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:18:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:18:57.580952 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3324aaa-3c9e-4709-aadc-cc8b8f968ac5 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:57.594672 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3324aaa-3c9e-4709-aadc-cc8b8f968ac5 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:57.602863 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c1b9cd-60e9-49aa-9a81-28c6093e6156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:57.603801 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c1b9cd-60e9-49aa-9a81-28c6093e6156 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:57.604142 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c1b9cd-60e9-49aa-9a81-28c6093e6156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:57.604432 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c1b9cd-60e9-49aa-9a81-28c6093e6156 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:57.608217 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c1b9cd-60e9-49aa-9a81-28c6093e6156 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:57.608217 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c1b9cd-60e9-49aa-9a81-28c6093e6156 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:57.650299 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 779/1543] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:18:57 2022] POST /identity/v3/projects => generated 353 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:18:57.654772 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc98c330-5397-41d1-9c4f-6f7bf6033f59 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:57.668989 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc98c330-5397-41d1-9c4f-6f7bf6033f59 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:57.689280 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1365019-ae06-4783-a4a8-c1576c9ed337 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:57.689529 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1365019-ae06-4783-a4a8-c1576c9ed337 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:57.689767 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1365019-ae06-4783-a4a8-c1576c9ed337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:57.722264 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e1365019-ae06-4783-a4a8-c1576c9ed337 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:18:57.723520 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e1365019-ae06-4783-a4a8-c1576c9ed337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:18:57.757286 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4659c7c-5034-435c-abfe-833064d82857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:57.757982 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4659c7c-5034-435c-abfe-833064d82857 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:57.758270 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4659c7c-5034-435c-abfe-833064d82857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:57.758577 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4659c7c-5034-435c-abfe-833064d82857 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:57.759495 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4659c7c-5034-435c-abfe-833064d82857 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:57.761468 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4659c7c-5034-435c-abfe-833064d82857 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:57.800109 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 780/1544] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:18:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:18:57.804550 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b26bba3-5849-48f3-8932-d54eb7a10c05 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:57.807767 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b26bba3-5849-48f3-8932-d54eb7a10c05 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:57.808660 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b26bba3-5849-48f3-8932-d54eb7a10c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:57.822386 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 765/1545] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:18:57 2022] POST /identity/v3/users => generated 277 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:18:57.828310 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4659c7c-5034-435c-abfe-833064d82857 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:57.837899 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7b26bba3-5849-48f3-8932-d54eb7a10c05 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:18:57.840632 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7b26bba3-5849-48f3-8932-d54eb7a10c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:18:57.844174 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4659c7c-5034-435c-abfe-833064d82857 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:57.854166 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 781/1546] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:18:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:18:57.854976 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7c07c2-9812-4f73-bfd5-4112608828b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:57.855719 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c07c2-9812-4f73-bfd5-4112608828b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:57.856741 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c07c2-9812-4f73-bfd5-4112608828b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:57.856741 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c07c2-9812-4f73-bfd5-4112608828b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:57.857677 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7c07c2-9812-4f73-bfd5-4112608828b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:57.858676 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b4638b-7de1-4fec-ae49-638700492f33 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:57.859191 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b4638b-7de1-4fec-ae49-638700492f33 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:57.859415 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7c07c2-9812-4f73-bfd5-4112608828b1 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:57.859909 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b4638b-7de1-4fec-ae49-638700492f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:57.877453 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 766/1547] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:18:57 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:18:57.880424 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446494db-f1d2-4389-8776-26a459faf5e9 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:57.880772 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446494db-f1d2-4389-8776-26a459faf5e9 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:57.881097 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446494db-f1d2-4389-8776-26a459faf5e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:57.897025 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c7b4638b-7de1-4fec-ae49-638700492f33 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:18:57.900272 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c7b4638b-7de1-4fec-ae49-638700492f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:18:57.915002 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-446494db-f1d2-4389-8776-26a459faf5e9 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:18:57.915765 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 782/1548] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:18:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:18:57.917590 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be76eb4-3d0e-49b1-aae8-a526be19651a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:57.918146 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be76eb4-3d0e-49b1-aae8-a526be19651a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:57.918672 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be76eb4-3d0e-49b1-aae8-a526be19651a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:57.919638 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-446494db-f1d2-4389-8776-26a459faf5e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:18:57.941168 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 767/1549] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:18:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:18:57.942179 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446494db-f1d2-4389-8776-26a459faf5e9 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:57.953054 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0be76eb4-3d0e-49b1-aae8-a526be19651a None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:18:57.954960 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0be76eb4-3d0e-49b1-aae8-a526be19651a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:18:57.957054 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446494db-f1d2-4389-8776-26a459faf5e9 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:57.967306 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cf5edb-898b-49e9-b1a6-b127261d6976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:57.968306 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf5edb-898b-49e9-b1a6-b127261d6976 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:57.969235 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf5edb-898b-49e9-b1a6-b127261d6976 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:57.970408 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf5edb-898b-49e9-b1a6-b127261d6976 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:57.971379 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cf5edb-898b-49e9-b1a6-b127261d6976 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:57.972925 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 783/1550] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:18:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:18:57.976223 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be76eb4-3d0e-49b1-aae8-a526be19651a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:57.976594 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cf5edb-898b-49e9-b1a6-b127261d6976 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:57.996661 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 768/1551] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:18:57 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:18:57.998405 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be76eb4-3d0e-49b1-aae8-a526be19651a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:57.998672 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f588ce-34db-4609-b4b0-c6744ed80df3 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:57.999309 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f588ce-34db-4609-b4b0-c6744ed80df3 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:57.999805 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f588ce-34db-4609-b4b0-c6744ed80df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:58.004866 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2d9874-2644-46d5-8ffc-8c8445c48ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:58.005909 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2d9874-2644-46d5-8ffc-8c8445c48ddf admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:58.006444 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2d9874-2644-46d5-8ffc-8c8445c48ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:58.007042 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2d9874-2644-46d5-8ffc-8c8445c48ddf admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:58.008002 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2d9874-2644-46d5-8ffc-8c8445c48ddf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:18:58.008645 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2d9874-2644-46d5-8ffc-8c8445c48ddf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:58.010093 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-2f2d9874-2644-46d5-8ffc-8c8445c48ddf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 04 09:18:58.010093 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Jul 04 09:18:58.010093 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: Jul 04 09:18:58.011628 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2d9874-2644-46d5-8ffc-8c8445c48ddf admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:58.019730 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-68f588ce-34db-4609-b4b0-c6744ed80df3 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:18:58.020940 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-68f588ce-34db-4609-b4b0-c6744ed80df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:18:58.027498 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 784/1552] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:18:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:18:58.029195 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866194b4-2f0c-4af9-993f-078cbaf13957 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:58.029715 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866194b4-2f0c-4af9-993f-078cbaf13957 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:58.029829 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 769/1553] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:18:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:18:58.030323 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866194b4-2f0c-4af9-993f-078cbaf13957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:58.030640 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f588ce-34db-4609-b4b0-c6744ed80df3 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:58.041474 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f588ce-34db-4609-b4b0-c6744ed80df3 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:58.046247 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4be7e2-398a-46e5-bee7-29c199e816c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:58.046924 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4be7e2-398a-46e5-bee7-29c199e816c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:58.047207 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4be7e2-398a-46e5-bee7-29c199e816c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:58.047484 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4be7e2-398a-46e5-bee7-29c199e816c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:58.048210 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4be7e2-398a-46e5-bee7-29c199e816c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:18:58.048516 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4be7e2-398a-46e5-bee7-29c199e816c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:58.050220 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4be7e2-398a-46e5-bee7-29c199e816c2 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:58.056405 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-866194b4-2f0c-4af9-993f-078cbaf13957 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:18:58.058091 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-866194b4-2f0c-4af9-993f-078cbaf13957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:18:58.064216 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 770/1554] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:18:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:18:58.065568 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f010f-0152-4601-acc5-83050aebd424 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:58.065860 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f010f-0152-4601-acc5-83050aebd424 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:58.066141 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f010f-0152-4601-acc5-83050aebd424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:58.069509 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 785/1555] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:18:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:18:58.070538 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866194b4-2f0c-4af9-993f-078cbaf13957 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:58.083723 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866194b4-2f0c-4af9-993f-078cbaf13957 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:58.085548 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-ef2f010f-0152-4601-acc5-83050aebd424 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:18:58.087285 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ef2f010f-0152-4601-acc5-83050aebd424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:18:58.089477 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add5887f-35d4-4eec-8328-61bdc740cc34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:58.090476 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add5887f-35d4-4eec-8328-61bdc740cc34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:58.090761 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add5887f-35d4-4eec-8328-61bdc740cc34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:58.091039 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add5887f-35d4-4eec-8328-61bdc740cc34 admin admin] PATH_INFO: `/v3/projects/35e6378c14874946aa7131a65a714d5e/users/983b3cf7bb4d4eb29e7cef371423db20/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:58.095681 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 771/1556] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:18:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:18:58.097885 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2f010f-0152-4601-acc5-83050aebd424 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:58.108149 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add5887f-35d4-4eec-8328-61bdc740cc34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35e6378c14874946aa7131a65a714d5e, user_id=983b3cf7bb4d4eb29e7cef371423db20, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:58.108402 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2f010f-0152-4601-acc5-83050aebd424 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:58.110944 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add5887f-35d4-4eec-8328-61bdc740cc34 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:58.113056 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c8a4f3-4ea9-4d09-b022-4beddc678e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:58.113749 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8a4f3-4ea9-4d09-b022-4beddc678e25 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:58.114046 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8a4f3-4ea9-4d09-b022-4beddc678e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:58.114325 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8a4f3-4ea9-4d09-b022-4beddc678e25 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:58.115091 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c8a4f3-4ea9-4d09-b022-4beddc678e25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:18:58.115359 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c8a4f3-4ea9-4d09-b022-4beddc678e25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:58.116959 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c8a4f3-4ea9-4d09-b022-4beddc678e25 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:58.125841 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 786/1557] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:18:58 2022] PUT /identity/v3/projects/35e6378c14874946aa7131a65a714d5e/users/983b3cf7bb4d4eb29e7cef371423db20/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:18:58.126740 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add5887f-35d4-4eec-8328-61bdc740cc34 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:58.140924 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 772/1558] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:18:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:18:58.140924 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89422e2f-4b5a-452a-8818-518424558e77 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:58.140924 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89422e2f-4b5a-452a-8818-518424558e77 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:58.140924 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89422e2f-4b5a-452a-8818-518424558e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:58.163708 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-89422e2f-4b5a-452a-8818-518424558e77 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:18:58.165547 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-89422e2f-4b5a-452a-8818-518424558e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:18:58.195281 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add5887f-35d4-4eec-8328-61bdc740cc34 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:58.200019 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8b4a34-f1d5-448b-8665-045a0c51b63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:58.200687 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8b4a34-f1d5-448b-8665-045a0c51b63c admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:58.200975 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8b4a34-f1d5-448b-8665-045a0c51b63c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:58.201331 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8b4a34-f1d5-448b-8665-045a0c51b63c admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:58.202096 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8b4a34-f1d5-448b-8665-045a0c51b63c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:58.204057 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8b4a34-f1d5-448b-8665-045a0c51b63c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:58.218024 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:18:58.232611 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 773/1559] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:18:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:18:58.236584 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89422e2f-4b5a-452a-8818-518424558e77 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:58.236750 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 787/1560] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:18:58 2022] POST /identity/v3/projects => generated 373 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:18:58.236750 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb4420-2c1a-4e1e-bda6-ad2a3a2dd44f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:58.236750 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb4420-2c1a-4e1e-bda6-ad2a3a2dd44f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:58.236750 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb4420-2c1a-4e1e-bda6-ad2a3a2dd44f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:58.241458 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:18:58.248574 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89422e2f-4b5a-452a-8818-518424558e77 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:58.255599 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d8cb4420-2c1a-4e1e-bda6-ad2a3a2dd44f None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:18:58.256797 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d8cb4420-2c1a-4e1e-bda6-ad2a3a2dd44f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:18:58.307876 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7f076a-30bd-490a-bde9-d902904e5330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:58.308554 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f076a-30bd-490a-bde9-d902904e5330 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:58.308827 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f076a-30bd-490a-bde9-d902904e5330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:58.309140 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f076a-30bd-490a-bde9-d902904e5330 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:58.309837 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7f076a-30bd-490a-bde9-d902904e5330 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:58.310869 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: WARNING py.warnings [None req-4e7f076a-30bd-490a-bde9-d902904e5330 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 09:18:58.310869 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: warnings.warn(msg) Jul 04 09:18:58.310869 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: Jul 04 09:18:58.312075 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7f076a-30bd-490a-bde9-d902904e5330 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:58.318324 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 788/1561] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:18:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:18:58.319615 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25049e01-ad0d-4f40-9e09-915e4eb068e2 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:58.319910 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25049e01-ad0d-4f40-9e09-915e4eb068e2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:58.320198 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25049e01-ad0d-4f40-9e09-915e4eb068e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:58.340053 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-25049e01-ad0d-4f40-9e09-915e4eb068e2 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:18:58.341008 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 774/1562] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:18:58 2022] POST /identity/v3/projects => generated 373 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:18:58.341302 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-25049e01-ad0d-4f40-9e09-915e4eb068e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:18:58.342314 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa3cb9e-6b2f-4cc6-9378-81e0dd2d2736 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:58.342628 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa3cb9e-6b2f-4cc6-9378-81e0dd2d2736 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:58.343531 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa3cb9e-6b2f-4cc6-9378-81e0dd2d2736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:58.362128 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-baa3cb9e-6b2f-4cc6-9378-81e0dd2d2736 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:18:58.363332 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-baa3cb9e-6b2f-4cc6-9378-81e0dd2d2736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:18:58.407228 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 789/1563] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:18:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:18:58.408611 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81088e8-d378-4bb5-b552-512613b37a67 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:58.408899 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81088e8-d378-4bb5-b552-512613b37a67 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:58.409209 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81088e8-d378-4bb5-b552-512613b37a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:58.429081 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 775/1564] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:18:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:18:58.429827 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa3cb9e-6b2f-4cc6-9378-81e0dd2d2736 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:58.436975 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c81088e8-d378-4bb5-b552-512613b37a67 None None] MFA Rules not processed for user `983b3cf7bb4d4eb29e7cef371423db20`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:18:58.438221 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c81088e8-d378-4bb5-b552-512613b37a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:18:58.439468 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa3cb9e-6b2f-4cc6-9378-81e0dd2d2736 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:58.444050 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5161eb6a-f576-4335-a42f-af41095a9b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:58.444705 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5161eb6a-f576-4335-a42f-af41095a9b2c admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:58.444989 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5161eb6a-f576-4335-a42f-af41095a9b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:58.445297 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5161eb6a-f576-4335-a42f-af41095a9b2c admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:58.446591 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5161eb6a-f576-4335-a42f-af41095a9b2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:58.448556 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5161eb6a-f576-4335-a42f-af41095a9b2c admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:58.474239 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 776/1565] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:18:58 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:18:58.475050 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5161eb6a-f576-4335-a42f-af41095a9b2c admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:58.505469 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 790/1566] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:18:58 2022] POST /identity/v3/auth/tokens => generated 3220 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:18:58.506806 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95af8e8-57bb-4799-9c8c-186faef15dea None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:58.507103 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95af8e8-57bb-4799-9c8c-186faef15dea None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:58.507383 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95af8e8-57bb-4799-9c8c-186faef15dea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:58.526056 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d95af8e8-57bb-4799-9c8c-186faef15dea None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:18:58.527262 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d95af8e8-57bb-4799-9c8c-186faef15dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:18:58.542174 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5161eb6a-f576-4335-a42f-af41095a9b2c admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:58.546008 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:18:58.546977 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620a274b-6644-4f94-a08d-0fbd4342a4ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:58.547938 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a274b-6644-4f94-a08d-0fbd4342a4ac admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:58.548219 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a274b-6644-4f94-a08d-0fbd4342a4ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:58.548536 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a274b-6644-4f94-a08d-0fbd4342a4ac admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:58.549269 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620a274b-6644-4f94-a08d-0fbd4342a4ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:58.551252 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620a274b-6644-4f94-a08d-0fbd4342a4ac admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:58.562581 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:18:58.576874 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 777/1567] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:18:58 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:18:58.577851 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620a274b-6644-4f94-a08d-0fbd4342a4ac admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:58.587986 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620a274b-6644-4f94-a08d-0fbd4342a4ac admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:58.592938 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21858d0-2073-41d2-a1de-390131f524f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:58.593701 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21858d0-2073-41d2-a1de-390131f524f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:58.594058 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21858d0-2073-41d2-a1de-390131f524f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:58.594486 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21858d0-2073-41d2-a1de-390131f524f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:58.595232 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21858d0-2073-41d2-a1de-390131f524f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:18:58.595624 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21858d0-2073-41d2-a1de-390131f524f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:58.595787 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 791/1568] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:18:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:18:58.599552 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95af8e8-57bb-4799-9c8c-186faef15dea None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:58.599692 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21858d0-2073-41d2-a1de-390131f524f9 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:58.607202 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95af8e8-57bb-4799-9c8c-186faef15dea None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:58.611520 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 778/1569] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:18:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:18:58.612246 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0abdb8-9855-434f-89fa-eccf75116981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:58.612772 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b143948-0130-4cc1-a76b-092de3cb13b9 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:58.613086 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b143948-0130-4cc1-a76b-092de3cb13b9 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:58.613396 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b143948-0130-4cc1-a76b-092de3cb13b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:58.613743 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0abdb8-9855-434f-89fa-eccf75116981 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:58.614187 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0abdb8-9855-434f-89fa-eccf75116981 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:58.614635 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0abdb8-9855-434f-89fa-eccf75116981 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:58.615345 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0abdb8-9855-434f-89fa-eccf75116981 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:58.616364 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-2a0abdb8-9855-434f-89fa-eccf75116981 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 09:18:58.616364 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Jul 04 09:18:58.616364 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: Jul 04 09:18:58.617623 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0abdb8-9855-434f-89fa-eccf75116981 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:58.631754 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-4b143948-0130-4cc1-a76b-092de3cb13b9 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:18:58.633070 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4b143948-0130-4cc1-a76b-092de3cb13b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:18:58.639117 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 792/1570] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:18:58 2022] POST /identity/v3/users => generated 287 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:18:58.639951 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0abdb8-9855-434f-89fa-eccf75116981 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:58.641445 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 779/1571] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:18:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:18:58.642350 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b143948-0130-4cc1-a76b-092de3cb13b9 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:58.650275 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0abdb8-9855-434f-89fa-eccf75116981 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:58.654474 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b143948-0130-4cc1-a76b-092de3cb13b9 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:58.655178 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f47d91c-210e-4d13-9851-d59a306abc7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:58.655861 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f47d91c-210e-4d13-9851-d59a306abc7a admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:58.656153 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f47d91c-210e-4d13-9851-d59a306abc7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:58.656431 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f47d91c-210e-4d13-9851-d59a306abc7a admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:58.657196 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f47d91c-210e-4d13-9851-d59a306abc7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:18:58.657510 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f47d91c-210e-4d13-9851-d59a306abc7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:58.659108 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f47d91c-210e-4d13-9851-d59a306abc7a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:58.659231 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6f49fe-f95f-4f37-9a05-c11979858cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:58.659814 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f49fe-f95f-4f37-9a05-c11979858cb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:58.660105 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f49fe-f95f-4f37-9a05-c11979858cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:58.660392 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f49fe-f95f-4f37-9a05-c11979858cb1 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:58.661321 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6f49fe-f95f-4f37-9a05-c11979858cb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:58.663748 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6f49fe-f95f-4f37-9a05-c11979858cb1 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:58.672778 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 793/1572] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:18:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:18:58.675143 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f47d91c-210e-4d13-9851-d59a306abc7a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:58.686565 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f47d91c-210e-4d13-9851-d59a306abc7a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:58.688879 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 780/1573] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:18:58 2022] POST /identity/v3/users => generated 287 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:18:58.689798 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6f49fe-f95f-4f37-9a05-c11979858cb1 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:58.691323 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68207b1-cbec-4b77-b911-311962d125ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:58.692033 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68207b1-cbec-4b77-b911-311962d125ca admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:58.692322 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68207b1-cbec-4b77-b911-311962d125ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:58.692599 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68207b1-cbec-4b77-b911-311962d125ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:58.693313 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68207b1-cbec-4b77-b911-311962d125ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:58.695235 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68207b1-cbec-4b77-b911-311962d125ca admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:58.700091 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6f49fe-f95f-4f37-9a05-c11979858cb1 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:58.704797 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356e43d6-af54-475c-a703-408074c224c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:58.705602 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e43d6-af54-475c-a703-408074c224c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:58.705891 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e43d6-af54-475c-a703-408074c224c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:58.706182 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e43d6-af54-475c-a703-408074c224c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:58.706917 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356e43d6-af54-475c-a703-408074c224c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:58.708514 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356e43d6-af54-475c-a703-408074c224c3 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:58.720750 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 781/1574] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:18:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:18:58.723792 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 794/1575] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:18:58 2022] POST /identity/v3/projects => generated 355 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:18:58.723792 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68207b1-cbec-4b77-b911-311962d125ca admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:58.724904 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356e43d6-af54-475c-a703-408074c224c3 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:58.735210 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356e43d6-af54-475c-a703-408074c224c3 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:58.785592 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68207b1-cbec-4b77-b911-311962d125ca admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:58.790790 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9c7ad4-a7be-4174-b6b2-633195e9fc3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:58.791620 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c7ad4-a7be-4174-b6b2-633195e9fc3f admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:58.791996 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c7ad4-a7be-4174-b6b2-633195e9fc3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:58.792301 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c7ad4-a7be-4174-b6b2-633195e9fc3f admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:58.792416 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f1741e-c2f6-4366-a433-32d786728b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:58.793032 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9c7ad4-a7be-4174-b6b2-633195e9fc3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:18:58.793151 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1741e-c2f6-4366-a433-32d786728b34 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:58.793367 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9c7ad4-a7be-4174-b6b2-633195e9fc3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:58.793495 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1741e-c2f6-4366-a433-32d786728b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:58.793660 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1741e-c2f6-4366-a433-32d786728b34 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:58.794356 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f1741e-c2f6-4366-a433-32d786728b34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:58.794953 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9c7ad4-a7be-4174-b6b2-633195e9fc3f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:58.796298 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f1741e-c2f6-4366-a433-32d786728b34 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:58.807469 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 795/1576] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:18:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:18:58.808329 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9c7ad4-a7be-4174-b6b2-633195e9fc3f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:58.817640 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9c7ad4-a7be-4174-b6b2-633195e9fc3f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:58.823251 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f4c9fc-f658-4741-aae3-3c73ce99890c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:58.823912 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f4c9fc-f658-4741-aae3-3c73ce99890c admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:58.824200 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f4c9fc-f658-4741-aae3-3c73ce99890c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:58.824480 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f4c9fc-f658-4741-aae3-3c73ce99890c admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:58.825270 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f4c9fc-f658-4741-aae3-3c73ce99890c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:58.826857 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f4c9fc-f658-4741-aae3-3c73ce99890c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:58.830453 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 782/1577] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:18:58 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:18:58.830453 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f1741e-c2f6-4366-a433-32d786728b34 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:58.838187 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 796/1578] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:18:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:18:58.839100 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f4c9fc-f658-4741-aae3-3c73ce99890c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:58.848379 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f4c9fc-f658-4741-aae3-3c73ce99890c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:58.896663 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f1741e-c2f6-4366-a433-32d786728b34 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:58.898505 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:18:58.902236 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9c029a-43da-4e4d-bdbf-1e153a46d464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:58.902957 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c029a-43da-4e4d-bdbf-1e153a46d464 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:58.903255 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c029a-43da-4e4d-bdbf-1e153a46d464 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:58.903726 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c029a-43da-4e4d-bdbf-1e153a46d464 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:58.904528 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9c029a-43da-4e4d-bdbf-1e153a46d464 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:18:58.904841 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9c029a-43da-4e4d-bdbf-1e153a46d464 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:58.906514 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9c029a-43da-4e4d-bdbf-1e153a46d464 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:58.907001 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4710883c-6d1f-4dd9-8003-ab4059a1256d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:58.908041 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710883c-6d1f-4dd9-8003-ab4059a1256d admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:58.908355 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710883c-6d1f-4dd9-8003-ab4059a1256d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:58.909090 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710883c-6d1f-4dd9-8003-ab4059a1256d admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:58.909373 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4710883c-6d1f-4dd9-8003-ab4059a1256d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:58.911261 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4710883c-6d1f-4dd9-8003-ab4059a1256d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:58.918749 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 783/1579] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:18:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:18:58.925168 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9c029a-43da-4e4d-bdbf-1e153a46d464 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:58.936804 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9c029a-43da-4e4d-bdbf-1e153a46d464 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:58.941687 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 797/1580] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:18:58 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:18:58.941687 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4710883c-6d1f-4dd9-8003-ab4059a1256d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:58.963856 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:18:59.002268 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4710883c-6d1f-4dd9-8003-ab4059a1256d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:59.005616 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a274f8c-49c1-453d-ab16-446f058fbd2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:59.006301 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a274f8c-49c1-453d-ab16-446f058fbd2f admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:59.006594 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a274f8c-49c1-453d-ab16-446f058fbd2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:59.006891 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a274f8c-49c1-453d-ab16-446f058fbd2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:59.007228 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e649c830-b9b4-400b-8739-a1f57e809e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:59.007627 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a274f8c-49c1-453d-ab16-446f058fbd2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:59.007897 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e649c830-b9b4-400b-8739-a1f57e809e6b admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:59.008183 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e649c830-b9b4-400b-8739-a1f57e809e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:59.008510 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e649c830-b9b4-400b-8739-a1f57e809e6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:59.009341 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a274f8c-49c1-453d-ab16-446f058fbd2f admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:59.009471 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e649c830-b9b4-400b-8739-a1f57e809e6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:59.010934 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e649c830-b9b4-400b-8739-a1f57e809e6b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:59.019697 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 784/1581] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:18:58 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:18:59.020570 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a274f8c-49c1-453d-ab16-446f058fbd2f admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:59.025341 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 798/1582] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:18:58 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:18:59.025341 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e649c830-b9b4-400b-8739-a1f57e809e6b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:59.031047 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a274f8c-49c1-453d-ab16-446f058fbd2f admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:59.032707 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e649c830-b9b4-400b-8739-a1f57e809e6b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:59.035685 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ff282-3849-44a8-ae1d-fd1a423a4076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:59.036403 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ff282-3849-44a8-ae1d-fd1a423a4076 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:59.036686 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ff282-3849-44a8-ae1d-fd1a423a4076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:59.036987 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ff282-3849-44a8-ae1d-fd1a423a4076 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:59.037432 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11191cc5-1632-4147-83eb-d36ec259bd01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:59.037688 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469ff282-3849-44a8-ae1d-fd1a423a4076 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:59.038065 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11191cc5-1632-4147-83eb-d36ec259bd01 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:59.038357 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11191cc5-1632-4147-83eb-d36ec259bd01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:59.038640 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11191cc5-1632-4147-83eb-d36ec259bd01 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:59.039323 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11191cc5-1632-4147-83eb-d36ec259bd01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:59.040371 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469ff282-3849-44a8-ae1d-fd1a423a4076 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:59.041953 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11191cc5-1632-4147-83eb-d36ec259bd01 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:59.064594 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 785/1583] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:18:59 2022] POST /identity/v3/users => generated 284 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:18:59.064594 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ff282-3849-44a8-ae1d-fd1a423a4076 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:59.071014 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 799/1584] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:18:59 2022] POST /identity/v3/projects => generated 389 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:18:59.071742 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ff282-3849-44a8-ae1d-fd1a423a4076 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:59.072051 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11191cc5-1632-4147-83eb-d36ec259bd01 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:59.130957 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa960a12-ef94-4614-a34b-176c4e1c81d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:59.131942 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa960a12-ef94-4614-a34b-176c4e1c81d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:59.132232 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa960a12-ef94-4614-a34b-176c4e1c81d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:59.132512 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa960a12-ef94-4614-a34b-176c4e1c81d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:59.133303 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa960a12-ef94-4614-a34b-176c4e1c81d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:59.135859 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa960a12-ef94-4614-a34b-176c4e1c81d1 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:59.136896 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11191cc5-1632-4147-83eb-d36ec259bd01 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:59.141725 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b372c699-160f-48e6-941e-7b7f9b32dc5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:59.142519 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372c699-160f-48e6-941e-7b7f9b32dc5a admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:59.142806 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372c699-160f-48e6-941e-7b7f9b32dc5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:59.143084 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372c699-160f-48e6-941e-7b7f9b32dc5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:59.143937 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b372c699-160f-48e6-941e-7b7f9b32dc5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:59.145601 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b372c699-160f-48e6-941e-7b7f9b32dc5a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:59.156376 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 786/1585] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:18:59 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:18:59.156694 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 800/1586] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:18:59.157422 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa960a12-ef94-4614-a34b-176c4e1c81d1 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:59.157890 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b372c699-160f-48e6-941e-7b7f9b32dc5a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:59.168593 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b372c699-160f-48e6-941e-7b7f9b32dc5a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:59.168857 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa960a12-ef94-4614-a34b-176c4e1c81d1 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:59.173140 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e606d1-8b9d-4f26-90a4-52237230ccf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:59.173425 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9072d5c1-3873-49d9-b725-5e2fcc2b106d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:59.173814 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e606d1-8b9d-4f26-90a4-52237230ccf2 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:59.174072 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9072d5c1-3873-49d9-b725-5e2fcc2b106d admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:59.174194 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e606d1-8b9d-4f26-90a4-52237230ccf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:59.174357 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9072d5c1-3873-49d9-b725-5e2fcc2b106d admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:59.174472 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e606d1-8b9d-4f26-90a4-52237230ccf2 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:59.174640 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9072d5c1-3873-49d9-b725-5e2fcc2b106d admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:59.175043 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e606d1-8b9d-4f26-90a4-52237230ccf2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:59.175323 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9072d5c1-3873-49d9-b725-5e2fcc2b106d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:59.176976 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e606d1-8b9d-4f26-90a4-52237230ccf2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:59.177242 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9072d5c1-3873-49d9-b725-5e2fcc2b106d admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:59.198267 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 801/1587] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:18:59 2022] POST /identity/v3/users => generated 293 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:18:59.199235 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e606d1-8b9d-4f26-90a4-52237230ccf2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:59.207635 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 787/1588] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:18:59 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:18:59.208635 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9072d5c1-3873-49d9-b725-5e2fcc2b106d admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:59.208772 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e606d1-8b9d-4f26-90a4-52237230ccf2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:59.239542 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:18:59.268649 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6654ee-c8c7-4104-950c-8ab66aa0a333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:59.269369 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6654ee-c8c7-4104-950c-8ab66aa0a333 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:59.269660 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6654ee-c8c7-4104-950c-8ab66aa0a333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:59.269942 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6654ee-c8c7-4104-950c-8ab66aa0a333 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:59.270701 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6654ee-c8c7-4104-950c-8ab66aa0a333 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:59.271746 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9072d5c1-3873-49d9-b725-5e2fcc2b106d admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:59.272343 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6654ee-c8c7-4104-950c-8ab66aa0a333 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:59.276436 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24810619-724c-4e63-b8d0-f8a26dd2b51a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:59.277372 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24810619-724c-4e63-b8d0-f8a26dd2b51a admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:59.278041 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24810619-724c-4e63-b8d0-f8a26dd2b51a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:59.278041 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24810619-724c-4e63-b8d0-f8a26dd2b51a admin admin] PATH_INFO: `/v3/projects/c31a26aa218c45aa88c4c7d3a737ecb0/users/bae0f0daed8447eea4f6bce81598b6ae/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:59.282636 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 802/1589] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:18:59.283520 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6654ee-c8c7-4104-950c-8ab66aa0a333 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:59.293010 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24810619-724c-4e63-b8d0-f8a26dd2b51a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c31a26aa218c45aa88c4c7d3a737ecb0, user_id=bae0f0daed8447eea4f6bce81598b6ae, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:59.293220 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6654ee-c8c7-4104-950c-8ab66aa0a333 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:59.294141 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: WARNING py.warnings [None req-24810619-724c-4e63-b8d0-f8a26dd2b51a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 09:18:59.294141 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: warnings.warn(msg) Jul 04 09:18:59.294141 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: Jul 04 09:18:59.296035 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24810619-724c-4e63-b8d0-f8a26dd2b51a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:59.297765 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc221fcf-0b9e-4789-893b-baeb59f8f093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:59.298410 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc221fcf-0b9e-4789-893b-baeb59f8f093 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:59.298693 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc221fcf-0b9e-4789-893b-baeb59f8f093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:59.298968 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc221fcf-0b9e-4789-893b-baeb59f8f093 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:59.299410 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:18:59.299882 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc221fcf-0b9e-4789-893b-baeb59f8f093 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:59.301267 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc221fcf-0b9e-4789-893b-baeb59f8f093 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:59.311442 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 788/1590] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:18:59 2022] PUT /identity/v3/projects/c31a26aa218c45aa88c4c7d3a737ecb0/users/bae0f0daed8447eea4f6bce81598b6ae/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:18:59.312385 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24810619-724c-4e63-b8d0-f8a26dd2b51a admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:59.312834 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 803/1591] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:18:59.313546 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc221fcf-0b9e-4789-893b-baeb59f8f093 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:59.323745 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc221fcf-0b9e-4789-893b-baeb59f8f093 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:59.377738 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24810619-724c-4e63-b8d0-f8a26dd2b51a admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:59.384317 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730661b8-50b9-48a5-9c58-6e6db3cd6f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:59.385530 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730661b8-50b9-48a5-9c58-6e6db3cd6f1c admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:59.386024 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730661b8-50b9-48a5-9c58-6e6db3cd6f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:59.386518 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730661b8-50b9-48a5-9c58-6e6db3cd6f1c admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:59.388027 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730661b8-50b9-48a5-9c58-6e6db3cd6f1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:59.390449 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730661b8-50b9-48a5-9c58-6e6db3cd6f1c admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:59.391791 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f91340-054c-4fb0-a615-00501196023d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:59.392489 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f91340-054c-4fb0-a615-00501196023d admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:59.392779 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f91340-054c-4fb0-a615-00501196023d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:59.393120 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f91340-054c-4fb0-a615-00501196023d admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:59.393855 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f91340-054c-4fb0-a615-00501196023d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:59.395569 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f91340-054c-4fb0-a615-00501196023d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:59.411465 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 804/1592] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:18:59.411465 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f91340-054c-4fb0-a615-00501196023d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:59.418151 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 789/1593] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:18:59 2022] POST /identity/v3/users => generated 295 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:18:59.418151 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730661b8-50b9-48a5-9c58-6e6db3cd6f1c admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:59.425262 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f91340-054c-4fb0-a615-00501196023d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:59.428027 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730661b8-50b9-48a5-9c58-6e6db3cd6f1c admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:59.432082 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f258f504-1d2b-469e-b6ea-c2011606ae16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:59.433191 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f258f504-1d2b-469e-b6ea-c2011606ae16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:59.433594 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f258f504-1d2b-469e-b6ea-c2011606ae16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:59.433884 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f258f504-1d2b-469e-b6ea-c2011606ae16 admin admin] PATH_INFO: `/v3/projects/d0e1400be01043d4b607d58b14c9d61d/users/d1ceafd31c9f443d972e31da5d9ff840/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:59.434296 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b194b986-bb2d-437e-81f7-43f2dfb1f782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:59.435103 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194b986-bb2d-437e-81f7-43f2dfb1f782 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:59.435395 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194b986-bb2d-437e-81f7-43f2dfb1f782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:59.435670 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194b986-bb2d-437e-81f7-43f2dfb1f782 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:59.436402 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b194b986-bb2d-437e-81f7-43f2dfb1f782 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:59.438061 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b194b986-bb2d-437e-81f7-43f2dfb1f782 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:59.450617 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 790/1594] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:18:59.451461 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b194b986-bb2d-437e-81f7-43f2dfb1f782 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:59.452152 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f258f504-1d2b-469e-b6ea-c2011606ae16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0e1400be01043d4b607d58b14c9d61d, user_id=d1ceafd31c9f443d972e31da5d9ff840, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:59.455198 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f258f504-1d2b-469e-b6ea-c2011606ae16 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:59.465266 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b194b986-bb2d-437e-81f7-43f2dfb1f782 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:59.469953 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 805/1595] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:18:59 2022] PUT /identity/v3/projects/d0e1400be01043d4b607d58b14c9d61d/users/d1ceafd31c9f443d972e31da5d9ff840/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:18:59.471064 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f258f504-1d2b-469e-b6ea-c2011606ae16 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:59.538325 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850e407e-63da-497e-a122-b7f1022bb899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:59.538585 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f258f504-1d2b-469e-b6ea-c2011606ae16 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:59.539442 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e407e-63da-497e-a122-b7f1022bb899 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:59.539981 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e407e-63da-497e-a122-b7f1022bb899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:59.540462 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e407e-63da-497e-a122-b7f1022bb899 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:59.541749 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850e407e-63da-497e-a122-b7f1022bb899 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:59.543446 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed73b4e0-a708-4be3-824e-bd31a7122194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:59.544341 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed73b4e0-a708-4be3-824e-bd31a7122194 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:59.544626 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed73b4e0-a708-4be3-824e-bd31a7122194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:59.544911 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed73b4e0-a708-4be3-824e-bd31a7122194 admin admin] PATH_INFO: `/v3/projects/65444bb2b7694e32a4acb48edcdadb5f/users/ab95cd8d0d6643d7835492a40acc8841/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:59.545309 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850e407e-63da-497e-a122-b7f1022bb899 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:59.562633 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed73b4e0-a708-4be3-824e-bd31a7122194 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65444bb2b7694e32a4acb48edcdadb5f, user_id=ab95cd8d0d6643d7835492a40acc8841, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:59.565656 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed73b4e0-a708-4be3-824e-bd31a7122194 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:59.572748 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 791/1596] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:18:59 2022] POST /identity/v3/users => generated 289 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:18:59.572748 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850e407e-63da-497e-a122-b7f1022bb899 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:59.581860 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 806/1597] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:18:59 2022] PUT /identity/v3/projects/65444bb2b7694e32a4acb48edcdadb5f/users/ab95cd8d0d6643d7835492a40acc8841/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:18:59.582191 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850e407e-63da-497e-a122-b7f1022bb899 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:59.587194 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c84205c-d189-437c-bc86-89e9aa844559 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:59.587816 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c84205c-d189-437c-bc86-89e9aa844559 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:59.587816 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c84205c-d189-437c-bc86-89e9aa844559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:59.619974 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:18:59.623620 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5c84205c-d189-437c-bc86-89e9aa844559 None None] MFA Rules not processed for user `bae0f0daed8447eea4f6bce81598b6ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:18:59.625277 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5c84205c-d189-437c-bc86-89e9aa844559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:18:59.656552 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f2f988-7ebf-4e9a-8bc7-09e61c785832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:59.657491 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f2f988-7ebf-4e9a-8bc7-09e61c785832 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:59.657782 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f2f988-7ebf-4e9a-8bc7-09e61c785832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:59.658084 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f2f988-7ebf-4e9a-8bc7-09e61c785832 admin admin] PATH_INFO: `/v3/projects/8b1abe013be34290b713c67f84aa0d1c/users/df4a0d9f18044b4485b2e85284a1d4d3/roles/f36e053b442540f08041ac7878760f65` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:59.667148 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:18:59.675512 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f2f988-7ebf-4e9a-8bc7-09e61c785832 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b1abe013be34290b713c67f84aa0d1c, user_id=df4a0d9f18044b4485b2e85284a1d4d3, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:59.678793 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f2f988-7ebf-4e9a-8bc7-09e61c785832 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:59.695491 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 792/1598] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:18:59 2022] PUT /identity/v3/projects/8b1abe013be34290b713c67f84aa0d1c/users/df4a0d9f18044b4485b2e85284a1d4d3/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:18:59.697168 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f2f988-7ebf-4e9a-8bc7-09e61c785832 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:59.701920 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 807/1599] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:18:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:18:59.702709 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c84205c-d189-437c-bc86-89e9aa844559 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:59.718317 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c84205c-d189-437c-bc86-89e9aa844559 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:59.784268 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f2f988-7ebf-4e9a-8bc7-09e61c785832 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:59.791089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedf2b88-1992-4a2e-ad81-0297a4926854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:59.791920 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedf2b88-1992-4a2e-ad81-0297a4926854 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:59.792601 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedf2b88-1992-4a2e-ad81-0297a4926854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:59.792887 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedf2b88-1992-4a2e-ad81-0297a4926854 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:59.793797 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedf2b88-1992-4a2e-ad81-0297a4926854 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:59.795507 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedf2b88-1992-4a2e-ad81-0297a4926854 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:59.798867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d15c57-864a-41a3-adca-3cb39ccf29ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:59.800065 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d15c57-864a-41a3-adca-3cb39ccf29ca admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:59.800358 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d15c57-864a-41a3-adca-3cb39ccf29ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:59.800658 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d15c57-864a-41a3-adca-3cb39ccf29ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:59.801967 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d15c57-864a-41a3-adca-3cb39ccf29ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:59.804755 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d15c57-864a-41a3-adca-3cb39ccf29ca admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:59.807802 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 793/1600] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:18:59.808736 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedf2b88-1992-4a2e-ad81-0297a4926854 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:59.818617 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedf2b88-1992-4a2e-ad81-0297a4926854 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:59.823687 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 808/1601] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:18:59.823687 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef29f1d7-eda3-4f9c-a069-9d3e13e32256 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:59.823687 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef29f1d7-eda3-4f9c-a069-9d3e13e32256 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:59.823687 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef29f1d7-eda3-4f9c-a069-9d3e13e32256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:59.824438 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccfaf59-1f0c-4f83-8225-9e3bfb9c4531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:59.825219 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccfaf59-1f0c-4f83-8225-9e3bfb9c4531 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:59.825499 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccfaf59-1f0c-4f83-8225-9e3bfb9c4531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:59.825781 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccfaf59-1f0c-4f83-8225-9e3bfb9c4531 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:59.826513 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccfaf59-1f0c-4f83-8225-9e3bfb9c4531 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:59.828133 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccfaf59-1f0c-4f83-8225-9e3bfb9c4531 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:59.838841 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 794/1602] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:18:59.839684 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccfaf59-1f0c-4f83-8225-9e3bfb9c4531 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:59.849750 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccfaf59-1f0c-4f83-8225-9e3bfb9c4531 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:59.854479 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35416935-6cd2-448b-a012-11565d295e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:59.855146 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35416935-6cd2-448b-a012-11565d295e3d admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:59.855438 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35416935-6cd2-448b-a012-11565d295e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:59.855720 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35416935-6cd2-448b-a012-11565d295e3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:59.856446 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35416935-6cd2-448b-a012-11565d295e3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:59.858099 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35416935-6cd2-448b-a012-11565d295e3d admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:59.861702 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ef29f1d7-eda3-4f9c-a069-9d3e13e32256 None None] MFA Rules not processed for user `d1ceafd31c9f443d972e31da5d9ff840`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:18:59.863397 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ef29f1d7-eda3-4f9c-a069-9d3e13e32256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:18:59.869918 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 795/1603] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:18:59.873835 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a467beba-43b1-4967-98ad-e2688ebeba47 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:59.873835 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a467beba-43b1-4967-98ad-e2688ebeba47 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:59.873835 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a467beba-43b1-4967-98ad-e2688ebeba47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:59.900529 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-a467beba-43b1-4967-98ad-e2688ebeba47 None None] MFA Rules not processed for user `ab95cd8d0d6643d7835492a40acc8841`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:18:59.901858 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a467beba-43b1-4967-98ad-e2688ebeba47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:18:59.944769 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 809/1604] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:18:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:18:59.950018 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef29f1d7-eda3-4f9c-a069-9d3e13e32256 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:59.959207 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef29f1d7-eda3-4f9c-a069-9d3e13e32256 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:59.965052 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db3c204-0795-43ad-b25f-c837f1bccf4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:59.965762 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db3c204-0795-43ad-b25f-c837f1bccf4f admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:59.966048 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db3c204-0795-43ad-b25f-c837f1bccf4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:59.966328 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db3c204-0795-43ad-b25f-c837f1bccf4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:59.967094 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db3c204-0795-43ad-b25f-c837f1bccf4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:59.968742 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db3c204-0795-43ad-b25f-c837f1bccf4f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:59.974645 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 796/1605] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:18:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:18:59.976078 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a860c13-c85b-48b5-a454-d6355169d1ea None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:59.976487 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a860c13-c85b-48b5-a454-d6355169d1ea None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:59.976671 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a860c13-c85b-48b5-a454-d6355169d1ea None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:59.978265 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 797/1606] 10.222.0.71 () {54 vars in 961 bytes} [Mon Jul 4 09:18:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:18:59.979127 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a860c13-c85b-48b5-a454-d6355169d1ea None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:59.980916 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 810/1607] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:18:59.981815 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db3c204-0795-43ad-b25f-c837f1bccf4f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:59.993958 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db3c204-0795-43ad-b25f-c837f1bccf4f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:59.998739 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a860c13-c85b-48b5-a454-d6355169d1ea None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:59.999561 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8f58ec-0761-4446-935f-9e1a48b1eeee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:00.000539 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f58ec-0761-4446-935f-9e1a48b1eeee admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:00.001023 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f58ec-0761-4446-935f-9e1a48b1eeee admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:00.001519 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f58ec-0761-4446-935f-9e1a48b1eeee admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:00.002527 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8f58ec-0761-4446-935f-9e1a48b1eeee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:00.003475 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ccd872-6c1b-424b-b409-25b35270c6a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:00.004508 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ccd872-6c1b-424b-b409-25b35270c6a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:00.004759 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8f58ec-0761-4446-935f-9e1a48b1eeee admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:00.004970 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ccd872-6c1b-424b-b409-25b35270c6a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:00.005281 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ccd872-6c1b-424b-b409-25b35270c6a7 admin admin] PATH_INFO: `/v3/projects/123c769553a1424e9c6dba4569bfaea5/users/e50f9b53a10a43fa9b5fec03d01f451f/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:00.017088 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 811/1608] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:19:00.018307 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8f58ec-0761-4446-935f-9e1a48b1eeee admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:00.029303 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8f58ec-0761-4446-935f-9e1a48b1eeee admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:00.032964 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ccd872-6c1b-424b-b409-25b35270c6a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=123c769553a1424e9c6dba4569bfaea5, user_id=e50f9b53a10a43fa9b5fec03d01f451f, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:00.035012 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aee7f26-35b9-4c1b-9eb8-d5caf35e5662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:00.036348 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ccd872-6c1b-424b-b409-25b35270c6a7 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:00.036472 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aee7f26-35b9-4c1b-9eb8-d5caf35e5662 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:00.036663 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aee7f26-35b9-4c1b-9eb8-d5caf35e5662 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:00.036951 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aee7f26-35b9-4c1b-9eb8-d5caf35e5662 admin admin] PATH_INFO: `/v3/projects/ba157b0e49f1447e82755035db8728d5/users/6bc30b7d536444008e5b5bab45c5c438/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:00.039456 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:00.057116 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 798/1609] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:18:59 2022] PUT /identity/v3/projects/123c769553a1424e9c6dba4569bfaea5/users/e50f9b53a10a43fa9b5fec03d01f451f/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:00.061368 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ccd872-6c1b-424b-b409-25b35270c6a7 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:00.061472 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aee7f26-35b9-4c1b-9eb8-d5caf35e5662 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba157b0e49f1447e82755035db8728d5, user_id=6bc30b7d536444008e5b5bab45c5c438, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:00.062002 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aee7f26-35b9-4c1b-9eb8-d5caf35e5662 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:00.078313 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 812/1610] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:19:00 2022] PUT /identity/v3/projects/ba157b0e49f1447e82755035db8728d5/users/6bc30b7d536444008e5b5bab45c5c438/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:00.082112 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aee7f26-35b9-4c1b-9eb8-d5caf35e5662 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:00.097923 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:00.141002 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ccd872-6c1b-424b-b409-25b35270c6a7 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:00.153352 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aee7f26-35b9-4c1b-9eb8-d5caf35e5662 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:00.159004 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216dcf40-f684-4894-b32e-fc38328263f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:00.160294 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216dcf40-f684-4894-b32e-fc38328263f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:00.160294 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216dcf40-f684-4894-b32e-fc38328263f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:00.160294 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216dcf40-f684-4894-b32e-fc38328263f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:00.160929 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216dcf40-f684-4894-b32e-fc38328263f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:00.162877 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216dcf40-f684-4894-b32e-fc38328263f9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:00.193742 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 813/1611] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:19:00 2022] POST /identity/v3/projects => generated 371 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:19:00.194577 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216dcf40-f684-4894-b32e-fc38328263f9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:00.211070 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad63591-eb3b-489c-bb52-27d9e0ad87bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:00.211898 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad63591-eb3b-489c-bb52-27d9e0ad87bd admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:00.212376 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad63591-eb3b-489c-bb52-27d9e0ad87bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:00.212670 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad63591-eb3b-489c-bb52-27d9e0ad87bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:00.213690 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad63591-eb3b-489c-bb52-27d9e0ad87bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:00.215337 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad63591-eb3b-489c-bb52-27d9e0ad87bd admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:00.227774 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 799/1612] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:19:00.229255 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc199c7-0490-436a-ac66-89c8a93e03b0 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:00.229555 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc199c7-0490-436a-ac66-89c8a93e03b0 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:00.229841 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc199c7-0490-436a-ac66-89c8a93e03b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:00.248974 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-afc199c7-0490-436a-ac66-89c8a93e03b0 None None] MFA Rules not processed for user `15039da209ec445d93d49e2bd6d854eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:00.251039 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-afc199c7-0490-436a-ac66-89c8a93e03b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:00.281299 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216dcf40-f684-4894-b32e-fc38328263f9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:00.285986 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76522456-0ebd-4d05-92e7-38381b8371fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:00.286882 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76522456-0ebd-4d05-92e7-38381b8371fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:00.287169 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76522456-0ebd-4d05-92e7-38381b8371fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:00.287447 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76522456-0ebd-4d05-92e7-38381b8371fd admin admin] PATH_INFO: `/v3/projects/8b1abe013be34290b713c67f84aa0d1c/users/df4a0d9f18044b4485b2e85284a1d4d3/roles/f36e053b442540f08041ac7878760f65` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:00.290393 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76522456-0ebd-4d05-92e7-38381b8371fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b1abe013be34290b713c67f84aa0d1c, user_id=df4a0d9f18044b4485b2e85284a1d4d3, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:00.293062 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76522456-0ebd-4d05-92e7-38381b8371fd admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:00.303516 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 814/1613] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:19:00 2022] PUT /identity/v3/projects/8b1abe013be34290b713c67f84aa0d1c/users/df4a0d9f18044b4485b2e85284a1d4d3/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:00.304842 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b46a8-4ee7-40d5-b3f4-4f5be803dfb0 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:00.305176 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b46a8-4ee7-40d5-b3f4-4f5be803dfb0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:00.306080 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b46a8-4ee7-40d5-b3f4-4f5be803dfb0 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:00.307667 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 815/1614] 10.222.0.71 () {54 vars in 961 bytes} [Mon Jul 4 09:19:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:19:00.308378 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002b46a8-4ee7-40d5-b3f4-4f5be803dfb0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:00.319912 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 800/1615] 10.222.0.71 () {58 vars in 1058 bytes} [Mon Jul 4 09:19:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:00.321177 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c201caf-0d8f-4768-a510-2db166d90bde None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:00.321522 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c201caf-0d8f-4768-a510-2db166d90bde None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:00.321864 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c201caf-0d8f-4768-a510-2db166d90bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:00.340133 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-0c201caf-0d8f-4768-a510-2db166d90bde None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:00.341405 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0c201caf-0d8f-4768-a510-2db166d90bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:00.379391 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002b46a8-4ee7-40d5-b3f4-4f5be803dfb0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:00.389673 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b615a3-4cbd-49e1-8d15-f9540f7b5ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:00.389953 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b615a3-4cbd-49e1-8d15-f9540f7b5ab7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:00.390242 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b615a3-4cbd-49e1-8d15-f9540f7b5ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:00.390522 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b615a3-4cbd-49e1-8d15-f9540f7b5ab7 admin admin] PATH_INFO: `/v3/projects/83d33ddd4b124161aba270998b7d2462/users/60d06d081a29460b8e85b0b301399f08/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:00.395296 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:00.409466 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b615a3-4cbd-49e1-8d15-f9540f7b5ab7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83d33ddd4b124161aba270998b7d2462, user_id=60d06d081a29460b8e85b0b301399f08, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:00.410931 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 801/1616] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:00.412230 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87bbc4-6126-468e-8d67-0d1867075faa None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:00.412355 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b615a3-4cbd-49e1-8d15-f9540f7b5ab7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:00.412526 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87bbc4-6126-468e-8d67-0d1867075faa None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:00.412800 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87bbc4-6126-468e-8d67-0d1867075faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:00.419420 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:00.429144 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 816/1617] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:19:00 2022] PUT /identity/v3/projects/83d33ddd4b124161aba270998b7d2462/users/60d06d081a29460b8e85b0b301399f08/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:00.429967 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b615a3-4cbd-49e1-8d15-f9540f7b5ab7 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:00.433276 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-4c87bbc4-6126-468e-8d67-0d1867075faa None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:00.434556 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4c87bbc4-6126-468e-8d67-0d1867075faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:00.499072 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b615a3-4cbd-49e1-8d15-f9540f7b5ab7 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:00.499259 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 802/1618] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:00.499259 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c87bbc4-6126-468e-8d67-0d1867075faa None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:00.504991 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c935859f-95b3-4969-bcb2-dcff3a9f7962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:00.505719 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935859f-95b3-4969-bcb2-dcff3a9f7962 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:00.506007 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935859f-95b3-4969-bcb2-dcff3a9f7962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:00.506223 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c87bbc4-6126-468e-8d67-0d1867075faa None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:00.506364 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935859f-95b3-4969-bcb2-dcff3a9f7962 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:00.507002 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c935859f-95b3-4969-bcb2-dcff3a9f7962 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:00.509684 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c935859f-95b3-4969-bcb2-dcff3a9f7962 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:00.511433 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd62383-f8a8-4570-905c-07ca8082a31d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:00.512328 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd62383-f8a8-4570-905c-07ca8082a31d admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:00.512610 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd62383-f8a8-4570-905c-07ca8082a31d admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:00.512887 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd62383-f8a8-4570-905c-07ca8082a31d admin admin] PATH_INFO: `/v3/projects/9acebc31809343e0abdc39ee5a127961/users/e034114ab8244cc99dbf977b458a43f9/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:00.529318 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd62383-f8a8-4570-905c-07ca8082a31d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9acebc31809343e0abdc39ee5a127961, user_id=e034114ab8244cc99dbf977b458a43f9, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:00.530931 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 817/1619] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:19:00 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:19:00.532003 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd62383-f8a8-4570-905c-07ca8082a31d admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:00.532159 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8fb645-9bf8-4257-95aa-507b6ef21ccd None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:00.532328 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8fb645-9bf8-4257-95aa-507b6ef21ccd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:00.532782 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8fb645-9bf8-4257-95aa-507b6ef21ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:00.546596 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 803/1620] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:19:00 2022] PUT /identity/v3/projects/9acebc31809343e0abdc39ee5a127961/users/e034114ab8244cc99dbf977b458a43f9/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:00.547889 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a65211-235d-48d1-99f3-4ff64f750393 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:00.548198 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a65211-235d-48d1-99f3-4ff64f750393 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:00.548459 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a65211-235d-48d1-99f3-4ff64f750393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:00.552301 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7c8fb645-9bf8-4257-95aa-507b6ef21ccd None None] MFA Rules not processed for user `15039da209ec445d93d49e2bd6d854eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:00.553550 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7c8fb645-9bf8-4257-95aa-507b6ef21ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:00.589969 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-02a65211-235d-48d1-99f3-4ff64f750393 None None] MFA Rules not processed for user `df4a0d9f18044b4485b2e85284a1d4d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:00.591236 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-02a65211-235d-48d1-99f3-4ff64f750393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:00.637647 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 818/1621] 10.222.0.71 () {58 vars in 1058 bytes} [Mon Jul 4 09:19:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:00.638528 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8fb645-9bf8-4257-95aa-507b6ef21ccd None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:00.650787 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8fb645-9bf8-4257-95aa-507b6ef21ccd None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:00.657461 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ad16a5-e04b-4c07-80f4-c9ebad7b68df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15039da209ec445d93d49e2bd6d854eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:00.658404 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ad16a5-e04b-4c07-80f4-c9ebad7b68df service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:00.658907 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ad16a5-e04b-4c07-80f4-c9ebad7b68df service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:00.659555 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ad16a5-e04b-4c07-80f4-c9ebad7b68df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:00.685658 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 804/1622] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:00 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:00.686873 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a65211-235d-48d1-99f3-4ff64f750393 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:00.730661 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ad16a5-e04b-4c07-80f4-c9ebad7b68df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:00.732655 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ad16a5-e04b-4c07-80f4-c9ebad7b68df service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:00.743100 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 819/1623] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:00 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:19:00.744729 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aefe4a-afe5-4473-80e2-5a8cb6f19549 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:00.745126 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aefe4a-afe5-4473-80e2-5a8cb6f19549 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:00.745485 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aefe4a-afe5-4473-80e2-5a8cb6f19549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:00.749776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a65211-235d-48d1-99f3-4ff64f750393 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:00.752411 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:00.753580 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef955235-a8d7-447e-9f0e-dccb23f2c8ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:00.754393 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef955235-a8d7-447e-9f0e-dccb23f2c8ec service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:00.754859 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef955235-a8d7-447e-9f0e-dccb23f2c8ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:00.755149 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef955235-a8d7-447e-9f0e-dccb23f2c8ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:00.767024 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a5aefe4a-afe5-4473-80e2-5a8cb6f19549 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:00.769173 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a5aefe4a-afe5-4473-80e2-5a8cb6f19549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:00.776061 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:00.830800 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef955235-a8d7-447e-9f0e-dccb23f2c8ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:00.832421 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef955235-a8d7-447e-9f0e-dccb23f2c8ec service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:00.845545 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 805/1624] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:19:00.846320 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef955235-a8d7-447e-9f0e-dccb23f2c8ec service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:00.855196 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef955235-a8d7-447e-9f0e-dccb23f2c8ec service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:00.859636 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f3f86b-3947-49ff-a186-f174cb9c0453 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:00.860287 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3f86b-3947-49ff-a186-f174cb9c0453 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:00.860575 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3f86b-3947-49ff-a186-f174cb9c0453 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:00.860859 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3f86b-3947-49ff-a186-f174cb9c0453 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:00.869181 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 820/1625] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:00.872040 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f3f86b-3947-49ff-a186-f174cb9c0453 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:00.873641 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f3f86b-3947-49ff-a186-f174cb9c0453 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:00.881549 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aefe4a-afe5-4473-80e2-5a8cb6f19549 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:00.887988 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 806/1626] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:19:00.889431 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d8bd9-f05d-4edd-8f53-32138c1df67f None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:00.889789 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d8bd9-f05d-4edd-8f53-32138c1df67f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:00.890142 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d8bd9-f05d-4edd-8f53-32138c1df67f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:00.912029 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-9c0d8bd9-f05d-4edd-8f53-32138c1df67f None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:00.912029 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-9c0d8bd9-f05d-4edd-8f53-32138c1df67f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:00.912300 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aefe4a-afe5-4473-80e2-5a8cb6f19549 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:00.924109 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 807/1627] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:00.925396 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0d8bd9-f05d-4edd-8f53-32138c1df67f None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:00.935451 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a20f217-51e5-4949-88ca-7439b83d9cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:00.936612 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a20f217-51e5-4949-88ca-7439b83d9cb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:00.936612 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a20f217-51e5-4949-88ca-7439b83d9cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:00.941559 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a20f217-51e5-4949-88ca-7439b83d9cb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:00.943666 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a20f217-51e5-4949-88ca-7439b83d9cb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:00.945523 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0d8bd9-f05d-4edd-8f53-32138c1df67f None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:00.951403 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a20f217-51e5-4949-88ca-7439b83d9cb8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:00.958386 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eded612-ad84-4caa-8668-e7a61e8b7a8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15039da209ec445d93d49e2bd6d854eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:00.959244 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eded612-ad84-4caa-8668-e7a61e8b7a8f service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:00.959535 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eded612-ad84-4caa-8668-e7a61e8b7a8f service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:00.959867 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eded612-ad84-4caa-8668-e7a61e8b7a8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:00.967589 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 821/1628] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:19:00.971558 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a20f217-51e5-4949-88ca-7439b83d9cb8 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:01.038400 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a20f217-51e5-4949-88ca-7439b83d9cb8 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:01.039422 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eded612-ad84-4caa-8668-e7a61e8b7a8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:01.042310 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eded612-ad84-4caa-8668-e7a61e8b7a8f service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:01.043570 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d0c986-e9cc-4874-8e35-a3d17cb404fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:01.044232 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0c986-e9cc-4874-8e35-a3d17cb404fa service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:01.044522 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0c986-e9cc-4874-8e35-a3d17cb404fa service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:01.044800 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0c986-e9cc-4874-8e35-a3d17cb404fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:01.052719 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 808/1629] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:00 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:19:01.053673 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eded612-ad84-4caa-8668-e7a61e8b7a8f service nova] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:01.055038 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0eded612-ad84-4caa-8668-e7a61e8b7a8f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:01.057241 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d0c986-e9cc-4874-8e35-a3d17cb404fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:01.058977 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d0c986-e9cc-4874-8e35-a3d17cb404fa service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:01.068361 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eded612-ad84-4caa-8668-e7a61e8b7a8f service nova] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:01.072910 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbfc217-bba4-4587-bc93-d3cdb7cee714 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:01.073747 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbfc217-bba4-4587-bc93-d3cdb7cee714 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:01.074040 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbfc217-bba4-4587-bc93-d3cdb7cee714 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:01.074325 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbfc217-bba4-4587-bc93-d3cdb7cee714 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:01.075190 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 822/1630] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:00 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:19:01.076236 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d0c986-e9cc-4874-8e35-a3d17cb404fa service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:01.087209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbfc217-bba4-4587-bc93-d3cdb7cee714 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:01.089103 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbfc217-bba4-4587-bc93-d3cdb7cee714 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:01.090809 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d0c986-e9cc-4874-8e35-a3d17cb404fa service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:01.095573 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1787197e-d286-4e96-a78d-ab00746e970a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:01.096378 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1787197e-d286-4e96-a78d-ab00746e970a service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:01.096807 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1787197e-d286-4e96-a78d-ab00746e970a service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:01.097249 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1787197e-d286-4e96-a78d-ab00746e970a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:01.100417 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 809/1631] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:19:01.101210 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbfc217-bba4-4587-bc93-d3cdb7cee714 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:01.112016 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1787197e-d286-4e96-a78d-ab00746e970a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:01.112872 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbfc217-bba4-4587-bc93-d3cdb7cee714 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:01.113670 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1787197e-d286-4e96-a78d-ab00746e970a service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:01.120199 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d65e608-703e-4922-a822-0dc2764c1890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:01.121231 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d65e608-703e-4922-a822-0dc2764c1890 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:01.121524 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d65e608-703e-4922-a822-0dc2764c1890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:01.122004 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d65e608-703e-4922-a822-0dc2764c1890 admin admin] PATH_INFO: `/v3/projects/6edc0208b3ca4ea49827030d358c315a/users/a1744b7b411346038ff2a0b6dc95807d/roles/f36e053b442540f08041ac7878760f65` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:01.128775 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 823/1632] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:19:01.130352 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d3f27-f785-4048-b3cd-50040f49fba5 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:01.130657 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d3f27-f785-4048-b3cd-50040f49fba5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:01.130986 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d3f27-f785-4048-b3cd-50040f49fba5 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:01.133109 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 824/1633] 10.222.0.71 () {54 vars in 912 bytes} [Mon Jul 4 09:19:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:19:01.141204 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab616b-7375-4f10-bd75-f2c08794b854 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:01.141204 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab616b-7375-4f10-bd75-f2c08794b854 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:01.141204 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab616b-7375-4f10-bd75-f2c08794b854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:01.146115 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d65e608-703e-4922-a822-0dc2764c1890 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6edc0208b3ca4ea49827030d358c315a, user_id=a1744b7b411346038ff2a0b6dc95807d, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:01.150027 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d65e608-703e-4922-a822-0dc2764c1890 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:01.179925 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 810/1634] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:19:01 2022] PUT /identity/v3/projects/6edc0208b3ca4ea49827030d358c315a/users/a1744b7b411346038ff2a0b6dc95807d/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:01.185758 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d65e608-703e-4922-a822-0dc2764c1890 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:01.198528 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-78ab616b-7375-4f10-bd75-f2c08794b854 None None] MFA Rules not processed for user `15039da209ec445d93d49e2bd6d854eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:01.203821 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-78ab616b-7375-4f10-bd75-f2c08794b854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:01.310480 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:01.323647 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d65e608-703e-4922-a822-0dc2764c1890 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:01.333875 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5216f428-b12b-4562-9fbb-5b9ca0e6748d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:01.335886 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5216f428-b12b-4562-9fbb-5b9ca0e6748d admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:01.338907 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5216f428-b12b-4562-9fbb-5b9ca0e6748d admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:01.340340 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5216f428-b12b-4562-9fbb-5b9ca0e6748d admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:01.342431 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5216f428-b12b-4562-9fbb-5b9ca0e6748d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:01.344840 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 825/1635] 10.222.0.71 () {58 vars in 1009 bytes} [Mon Jul 4 09:19:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:19:01.344971 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5216f428-b12b-4562-9fbb-5b9ca0e6748d admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:01.345925 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ab616b-7375-4f10-bd75-f2c08794b854 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:01.379027 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 811/1636] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:19:01.379027 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5216f428-b12b-4562-9fbb-5b9ca0e6748d admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:01.387110 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:01.482925 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ab616b-7375-4f10-bd75-f2c08794b854 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:01.494016 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a0d003-8822-43fd-b300-f9f6fd065635 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:01.494725 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0d003-8822-43fd-b300-f9f6fd065635 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:01.495010 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0d003-8822-43fd-b300-f9f6fd065635 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:01.495288 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0d003-8822-43fd-b300-f9f6fd065635 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:01.536364 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5216f428-b12b-4562-9fbb-5b9ca0e6748d admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:01.544948 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78029c8b-ba55-4e0b-a5af-d3c60e918b57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:01.546039 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78029c8b-ba55-4e0b-a5af-d3c60e918b57 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:01.546400 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78029c8b-ba55-4e0b-a5af-d3c60e918b57 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:01.546773 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78029c8b-ba55-4e0b-a5af-d3c60e918b57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:01.564256 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78029c8b-ba55-4e0b-a5af-d3c60e918b57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:01.566221 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78029c8b-ba55-4e0b-a5af-d3c60e918b57 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:01.583042 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 812/1637] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:19:01.594255 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78029c8b-ba55-4e0b-a5af-d3c60e918b57 service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:01.601283 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a0d003-8822-43fd-b300-f9f6fd065635 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:01.603866 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a0d003-8822-43fd-b300-f9f6fd065635 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:01.629560 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78029c8b-ba55-4e0b-a5af-d3c60e918b57 service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:01.639750 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84b7c11-547e-4e08-b18b-fbf29f476c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:01.640934 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b7c11-547e-4e08-b18b-fbf29f476c29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:01.641836 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b7c11-547e-4e08-b18b-fbf29f476c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:01.642539 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b7c11-547e-4e08-b18b-fbf29f476c29 admin admin] PATH_INFO: `/v3/projects/6edc0208b3ca4ea49827030d358c315a/users/a1744b7b411346038ff2a0b6dc95807d/roles/f36e053b442540f08041ac7878760f65` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:01.645536 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 826/1638] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:01 2022] GET /identity/v3/auth/tokens => generated 3220 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:19:01.646298 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84b7c11-547e-4e08-b18b-fbf29f476c29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6edc0208b3ca4ea49827030d358c315a, user_id=a1744b7b411346038ff2a0b6dc95807d, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:01.653026 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3baedb7-4f18-41f4-9284-3203ad33eba4 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:01.653345 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3baedb7-4f18-41f4-9284-3203ad33eba4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:01.653628 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3baedb7-4f18-41f4-9284-3203ad33eba4 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:01.663461 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 827/1639] 10.222.0.71 () {54 vars in 912 bytes} [Mon Jul 4 09:19:01 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:19:01.666497 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84b7c11-547e-4e08-b18b-fbf29f476c29 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:01.669685 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211ea320-5f41-4150-a79f-4ce41eb188db None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:01.669977 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211ea320-5f41-4150-a79f-4ce41eb188db None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:01.672328 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211ea320-5f41-4150-a79f-4ce41eb188db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:01.695295 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 813/1640] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:19:01 2022] PUT /identity/v3/projects/6edc0208b3ca4ea49827030d358c315a/users/a1744b7b411346038ff2a0b6dc95807d/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:01.698124 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d979c5-d294-4ab7-b085-e56c51137d40 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:01.698548 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d979c5-d294-4ab7-b085-e56c51137d40 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:01.698967 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d979c5-d294-4ab7-b085-e56c51137d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:01.736367 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-211ea320-5f41-4150-a79f-4ce41eb188db None None] MFA Rules not processed for user `15039da209ec445d93d49e2bd6d854eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:01.737628 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-211ea320-5f41-4150-a79f-4ce41eb188db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:01.773776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-d9d979c5-d294-4ab7-b085-e56c51137d40 None None] MFA Rules not processed for user `a1744b7b411346038ff2a0b6dc95807d`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:01.775594 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d9d979c5-d294-4ab7-b085-e56c51137d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:01.952628 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 814/1641] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:01 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:01.964132 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 828/1642] 10.222.0.71 () {58 vars in 1009 bytes} [Mon Jul 4 09:19:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:02.156231 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d979c5-d294-4ab7-b085-e56c51137d40 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:02.350636 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d979c5-d294-4ab7-b085-e56c51137d40 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:02.359838 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc7a96-d82c-4f60-9051-3d079293a235 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:02.362392 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc7a96-d82c-4f60-9051-3d079293a235 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:02.363276 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc7a96-d82c-4f60-9051-3d079293a235 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:02.363276 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc7a96-d82c-4f60-9051-3d079293a235 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:02.396088 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc7a96-d82c-4f60-9051-3d079293a235 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:02.396241 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211ea320-5f41-4150-a79f-4ce41eb188db None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:02.398714 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc7a96-d82c-4f60-9051-3d079293a235 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:02.412954 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211ea320-5f41-4150-a79f-4ce41eb188db None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:02.417598 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c019cd-ae23-4aaf-a7f7-4b8b53f862f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:02.418156 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 815/1643] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:02 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:19:02.418282 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c019cd-ae23-4aaf-a7f7-4b8b53f862f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:02.418776 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c019cd-ae23-4aaf-a7f7-4b8b53f862f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:02.418901 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c019cd-ae23-4aaf-a7f7-4b8b53f862f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:02.441240 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c019cd-ae23-4aaf-a7f7-4b8b53f862f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:02.442923 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c019cd-ae23-4aaf-a7f7-4b8b53f862f4 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:02.470865 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 829/1644] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:19:03.015813 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc7a96-d82c-4f60-9051-3d079293a235 service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:03.032972 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:03.176187 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc7a96-d82c-4f60-9051-3d079293a235 service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:03.184411 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8791ef-657d-4ae9-af4f-534f372935a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:03.185477 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8791ef-657d-4ae9-af4f-534f372935a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:03.185768 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8791ef-657d-4ae9-af4f-534f372935a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:03.186200 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8791ef-657d-4ae9-af4f-534f372935a5 admin admin] PATH_INFO: `/v3/users/ab95cd8d0d6643d7835492a40acc8841` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:03.187617 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8791ef-657d-4ae9-af4f-534f372935a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab95cd8d0d6643d7835492a40acc8841)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:03.188810 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: WARNING py.warnings [None req-9f8791ef-657d-4ae9-af4f-534f372935a5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 09:19:03.188810 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: warnings.warn(msg) Jul 04 09:19:03.188810 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: Jul 04 09:19:03.207314 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8791ef-657d-4ae9-af4f-534f372935a5 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:03.343401 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-9f8791ef-657d-4ae9-af4f-534f372935a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab95cd8d0d6643d7835492a40acc8841'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:03.357027 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-9f8791ef-657d-4ae9-af4f-534f372935a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab95cd8d0d6643d7835492a40acc8841'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:03.360125 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-9f8791ef-657d-4ae9-af4f-534f372935a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab95cd8d0d6643d7835492a40acc8841'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:03.395682 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-9f8791ef-657d-4ae9-af4f-534f372935a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab95cd8d0d6643d7835492a40acc8841'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:03.435724 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-9f8791ef-657d-4ae9-af4f-534f372935a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab95cd8d0d6643d7835492a40acc8841'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:03.450748 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 816/1645] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:19:03 2022] DELETE /identity/v3/users/ab95cd8d0d6643d7835492a40acc8841 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:19:03.455857 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb7266-3c05-4b6b-b18b-30ebd1ca5ab0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:03.456408 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb7266-3c05-4b6b-b18b-30ebd1ca5ab0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:03.456506 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb7266-3c05-4b6b-b18b-30ebd1ca5ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:03.490534 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4afb7266-3c05-4b6b-b18b-30ebd1ca5ab0 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:03.494528 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4afb7266-3c05-4b6b-b18b-30ebd1ca5ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:03.598987 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:03.629781 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 830/1646] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:03.645182 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8791ef-657d-4ae9-af4f-534f372935a5 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:03.646791 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-9f8791ef-657d-4ae9-af4f-534f372935a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:03.700565 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afb7266-3c05-4b6b-b18b-30ebd1ca5ab0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:03.702405 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4afb7266-3c05-4b6b-b18b-30ebd1ca5ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:03.722719 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afb7266-3c05-4b6b-b18b-30ebd1ca5ab0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:03.738856 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bbafc1-1133-4e7f-8164-c02ac54f8594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:03.739748 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bbafc1-1133-4e7f-8164-c02ac54f8594 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:03.740026 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bbafc1-1133-4e7f-8164-c02ac54f8594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:03.740302 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bbafc1-1133-4e7f-8164-c02ac54f8594 admin admin] PATH_INFO: `/v3/users/d1ceafd31c9f443d972e31da5d9ff840` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:03.741967 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bbafc1-1133-4e7f-8164-c02ac54f8594 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1ceafd31c9f443d972e31da5d9ff840)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:03.747623 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bbafc1-1133-4e7f-8164-c02ac54f8594 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:03.754475 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8791ef-657d-4ae9-af4f-534f372935a5 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:03.760562 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5458a7d3-6c3e-4f30-943b-d8393d3bdd82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:03.762056 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5458a7d3-6c3e-4f30-943b-d8393d3bdd82 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:03.762720 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5458a7d3-6c3e-4f30-943b-d8393d3bdd82 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:03.763206 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5458a7d3-6c3e-4f30-943b-d8393d3bdd82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:03.777103 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5458a7d3-6c3e-4f30-943b-d8393d3bdd82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:03.779337 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5458a7d3-6c3e-4f30-943b-d8393d3bdd82 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:03.803096 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 817/1647] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:19:03.859858 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d8bbafc1-1133-4e7f-8164-c02ac54f8594 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1ceafd31c9f443d972e31da5d9ff840'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:03.860735 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d8bbafc1-1133-4e7f-8164-c02ac54f8594 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1ceafd31c9f443d972e31da5d9ff840'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:03.877334 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d8bbafc1-1133-4e7f-8164-c02ac54f8594 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1ceafd31c9f443d972e31da5d9ff840'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:03.894766 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d8bbafc1-1133-4e7f-8164-c02ac54f8594 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1ceafd31c9f443d972e31da5d9ff840'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:03.895706 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d8bbafc1-1133-4e7f-8164-c02ac54f8594 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1ceafd31c9f443d972e31da5d9ff840'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:03.912932 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 831/1648] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:19:03 2022] DELETE /identity/v3/users/d1ceafd31c9f443d972e31da5d9ff840 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:19:03.919282 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1854bc-c218-4eb8-ad68-88cf9d6f20cf None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:03.919955 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1854bc-c218-4eb8-ad68-88cf9d6f20cf None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:03.920381 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1854bc-c218-4eb8-ad68-88cf9d6f20cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:03.942634 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-ff1854bc-c218-4eb8-ad68-88cf9d6f20cf None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:03.944252 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ff1854bc-c218-4eb8-ad68-88cf9d6f20cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:03.982873 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:04.071303 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 818/1649] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:19:04.087233 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bbafc1-1133-4e7f-8164-c02ac54f8594 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:04.089671 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d8bbafc1-1133-4e7f-8164-c02ac54f8594 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:04.342123 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bbafc1-1133-4e7f-8164-c02ac54f8594 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:04.361275 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152077aa-925c-4001-aca3-86553b77b4f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:04.361978 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152077aa-925c-4001-aca3-86553b77b4f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:04.362240 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152077aa-925c-4001-aca3-86553b77b4f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:04.362594 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152077aa-925c-4001-aca3-86553b77b4f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:04.396175 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152077aa-925c-4001-aca3-86553b77b4f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:04.398075 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152077aa-925c-4001-aca3-86553b77b4f0 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:04.429407 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 832/1650] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:19:04.531871 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1854bc-c218-4eb8-ad68-88cf9d6f20cf None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:04.533615 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ff1854bc-c218-4eb8-ad68-88cf9d6f20cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:04.567216 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1854bc-c218-4eb8-ad68-88cf9d6f20cf None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:04.595752 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47c0380-f4da-4237-aa38-5a7c05856ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:04.597029 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c0380-f4da-4237-aa38-5a7c05856ecc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:04.599788 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c0380-f4da-4237-aa38-5a7c05856ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:04.600066 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c0380-f4da-4237-aa38-5a7c05856ecc admin admin] PATH_INFO: `/v3/projects/65444bb2b7694e32a4acb48edcdadb5f` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:04.602372 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47c0380-f4da-4237-aa38-5a7c05856ecc admin admin] RBAC: Authorizing `identity:delete_project(project_id=65444bb2b7694e32a4acb48edcdadb5f)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:04.605312 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47c0380-f4da-4237-aa38-5a7c05856ecc admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:04.745310 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b47c0380-f4da-4237-aa38-5a7c05856ecc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65444bb2b7694e32a4acb48edcdadb5f', 'request_id': 'req-b47c0380-f4da-4237-aa38-5a7c05856ecc'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:04.755979 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b47c0380-f4da-4237-aa38-5a7c05856ecc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65444bb2b7694e32a4acb48edcdadb5f', 'request_id': 'req-b47c0380-f4da-4237-aa38-5a7c05856ecc'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:04.769599 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b47c0380-f4da-4237-aa38-5a7c05856ecc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65444bb2b7694e32a4acb48edcdadb5f', 'request_id': 'req-b47c0380-f4da-4237-aa38-5a7c05856ecc'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:04.793271 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b47c0380-f4da-4237-aa38-5a7c05856ecc admin admin] The token cache is being invalidate because project 65444bb2b7694e32a4acb48edcdadb5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:19:04.793673 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b47c0380-f4da-4237-aa38-5a7c05856ecc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:04.794295 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b47c0380-f4da-4237-aa38-5a7c05856ecc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:04.809626 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 819/1651] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:19:04 2022] DELETE /identity/v3/projects/65444bb2b7694e32a4acb48edcdadb5f => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:04.814547 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9455592c-ef99-4762-bc33-662ae1d3564f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:04.815101 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9455592c-ef99-4762-bc33-662ae1d3564f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:04.815496 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9455592c-ef99-4762-bc33-662ae1d3564f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:04.883682 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9455592c-ef99-4762-bc33-662ae1d3564f None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:04.894863 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9455592c-ef99-4762-bc33-662ae1d3564f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:05.076345 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 833/1652] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:05.080065 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a522f4-c5b6-42ed-bb75-c1a2be8fbc98 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:05.080546 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a522f4-c5b6-42ed-bb75-c1a2be8fbc98 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:05.080644 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a522f4-c5b6-42ed-bb75-c1a2be8fbc98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:05.134696 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-c2a522f4-c5b6-42ed-bb75-c1a2be8fbc98 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:05.139833 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c2a522f4-c5b6-42ed-bb75-c1a2be8fbc98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:05.157897 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 820/1653] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:05.161745 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9455592c-ef99-4762-bc33-662ae1d3564f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:05.169180 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:05.173166 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9455592c-ef99-4762-bc33-662ae1d3564f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:05.264101 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce38d0d-ead5-4d1f-8a83-69a9daef08c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:05.264101 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce38d0d-ead5-4d1f-8a83-69a9daef08c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:05.264101 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce38d0d-ead5-4d1f-8a83-69a9daef08c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:05.264101 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce38d0d-ead5-4d1f-8a83-69a9daef08c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:05.265854 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce38d0d-ead5-4d1f-8a83-69a9daef08c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:19:05.265854 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce38d0d-ead5-4d1f-8a83-69a9daef08c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:05.265854 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce38d0d-ead5-4d1f-8a83-69a9daef08c0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:05.265854 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 834/1654] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:19:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:19:05.266328 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a522f4-c5b6-42ed-bb75-c1a2be8fbc98 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:05.330125 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a522f4-c5b6-42ed-bb75-c1a2be8fbc98 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:05.330125 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e747fdb3-a8ff-4513-a1b2-8d31bf719acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:05.330125 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e747fdb3-a8ff-4513-a1b2-8d31bf719acb admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:05.330125 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e747fdb3-a8ff-4513-a1b2-8d31bf719acb admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:05.331272 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e747fdb3-a8ff-4513-a1b2-8d31bf719acb admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:05.331272 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e747fdb3-a8ff-4513-a1b2-8d31bf719acb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:05.331272 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e747fdb3-a8ff-4513-a1b2-8d31bf719acb admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:05.420899 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 821/1655] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:19:05 2022] POST /identity/v3/projects => generated 381 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:19:05.425027 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce38d0d-ead5-4d1f-8a83-69a9daef08c0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:05.453868 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce38d0d-ead5-4d1f-8a83-69a9daef08c0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:05.498838 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e747fdb3-a8ff-4513-a1b2-8d31bf719acb admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:05.503156 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e747fdb3-a8ff-4513-a1b2-8d31bf719acb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:05.622262 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8578a34a-d729-4f36-b25c-a8681218f9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:05.623303 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8578a34a-d729-4f36-b25c-a8681218f9ff admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:05.624972 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8578a34a-d729-4f36-b25c-a8681218f9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:05.625531 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8578a34a-d729-4f36-b25c-a8681218f9ff admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:05.627448 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8578a34a-d729-4f36-b25c-a8681218f9ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:05.629898 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8578a34a-d729-4f36-b25c-a8681218f9ff admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:05.686311 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 835/1656] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:19:05 2022] POST /identity/v3/users => generated 291 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:19:05.688302 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8578a34a-d729-4f36-b25c-a8681218f9ff admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:05.691198 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8578a34a-d729-4f36-b25c-a8681218f9ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:05.702867 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e747fdb3-a8ff-4513-a1b2-8d31bf719acb admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:05.716828 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5878b4-6151-4f5e-9742-57a4e96c8d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:05.718051 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5878b4-6151-4f5e-9742-57a4e96c8d10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:05.718992 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5878b4-6151-4f5e-9742-57a4e96c8d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:05.719095 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8578a34a-d729-4f36-b25c-a8681218f9ff admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:05.719197 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5878b4-6151-4f5e-9742-57a4e96c8d10 admin admin] PATH_INFO: `/v3/projects/d0e1400be01043d4b607d58b14c9d61d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:05.721834 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5878b4-6151-4f5e-9742-57a4e96c8d10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0e1400be01043d4b607d58b14c9d61d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:05.726267 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9d2d9c-9487-4c43-a7b4-93c7ed84f531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:05.728214 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9d2d9c-9487-4c43-a7b4-93c7ed84f531 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:05.729370 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9d2d9c-9487-4c43-a7b4-93c7ed84f531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:05.729974 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9d2d9c-9487-4c43-a7b4-93c7ed84f531 admin admin] PATH_INFO: `/v3/users/bae0f0daed8447eea4f6bce81598b6ae` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:05.731016 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5878b4-6151-4f5e-9742-57a4e96c8d10 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:05.732143 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9d2d9c-9487-4c43-a7b4-93c7ed84f531 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bae0f0daed8447eea4f6bce81598b6ae)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:05.734875 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9d2d9c-9487-4c43-a7b4-93c7ed84f531 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:05.806325 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:05.834505 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ed9d2d9c-9487-4c43-a7b4-93c7ed84f531 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bae0f0daed8447eea4f6bce81598b6ae'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:05.839786 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ed9d2d9c-9487-4c43-a7b4-93c7ed84f531 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bae0f0daed8447eea4f6bce81598b6ae'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:05.848544 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-9a5878b4-6151-4f5e-9742-57a4e96c8d10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0e1400be01043d4b607d58b14c9d61d', 'request_id': 'req-9a5878b4-6151-4f5e-9742-57a4e96c8d10'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:05.852658 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ed9d2d9c-9487-4c43-a7b4-93c7ed84f531 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bae0f0daed8447eea4f6bce81598b6ae'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:05.864072 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-9a5878b4-6151-4f5e-9742-57a4e96c8d10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0e1400be01043d4b607d58b14c9d61d', 'request_id': 'req-9a5878b4-6151-4f5e-9742-57a4e96c8d10'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:05.879331 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-9a5878b4-6151-4f5e-9742-57a4e96c8d10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0e1400be01043d4b607d58b14c9d61d', 'request_id': 'req-9a5878b4-6151-4f5e-9742-57a4e96c8d10'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:05.879910 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ed9d2d9c-9487-4c43-a7b4-93c7ed84f531 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bae0f0daed8447eea4f6bce81598b6ae'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:05.881351 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ed9d2d9c-9487-4c43-a7b4-93c7ed84f531 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bae0f0daed8447eea4f6bce81598b6ae'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:05.906365 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 836/1657] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:19:05 2022] DELETE /identity/v3/users/bae0f0daed8447eea4f6bce81598b6ae => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:05.906365 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9d2d9c-9487-4c43-a7b4-93c7ed84f531 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:05.907615 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ed9d2d9c-9487-4c43-a7b4-93c7ed84f531 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:05.910051 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-9a5878b4-6151-4f5e-9742-57a4e96c8d10 admin admin] The token cache is being invalidate because project d0e1400be01043d4b607d58b14c9d61d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:19:05.910352 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-9a5878b4-6151-4f5e-9742-57a4e96c8d10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:05.917701 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-9a5878b4-6151-4f5e-9742-57a4e96c8d10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:05.922622 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 822/1658] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:19:05 2022] DELETE /identity/v3/projects/d0e1400be01043d4b607d58b14c9d61d => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:05.924057 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a7765-e428-4f49-9cf3-b126043d2ed8 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:05.924774 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a7765-e428-4f49-9cf3-b126043d2ed8 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:05.924774 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a7765-e428-4f49-9cf3-b126043d2ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:05.966851 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-b51a7765-e428-4f49-9cf3-b126043d2ed8 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:05.972976 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b51a7765-e428-4f49-9cf3-b126043d2ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:06.039811 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:06.067144 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9d2d9c-9487-4c43-a7b4-93c7ed84f531 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:06.081031 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f397b227-0c19-49e5-9e8f-9b5fcfdc8708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:06.081910 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397b227-0c19-49e5-9e8f-9b5fcfdc8708 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:06.082312 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397b227-0c19-49e5-9e8f-9b5fcfdc8708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:06.082704 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397b227-0c19-49e5-9e8f-9b5fcfdc8708 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:06.083563 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f397b227-0c19-49e5-9e8f-9b5fcfdc8708 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:06.085436 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f397b227-0c19-49e5-9e8f-9b5fcfdc8708 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:06.133821 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 823/1659] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:06.135480 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a7765-e428-4f49-9cf3-b126043d2ed8 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:06.137570 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 837/1660] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:05 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:19:06.138412 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b51a7765-e428-4f49-9cf3-b126043d2ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:06.138890 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f397b227-0c19-49e5-9e8f-9b5fcfdc8708 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:06.140785 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f397b227-0c19-49e5-9e8f-9b5fcfdc8708 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:06.156057 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f397b227-0c19-49e5-9e8f-9b5fcfdc8708 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:06.161680 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a087c38b-3fc0-48da-a73c-4194b2444ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:06.162530 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a087c38b-3fc0-48da-a73c-4194b2444ed6 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:06.163230 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a087c38b-3fc0-48da-a73c-4194b2444ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:06.163230 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a087c38b-3fc0-48da-a73c-4194b2444ed6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:06.164124 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a087c38b-3fc0-48da-a73c-4194b2444ed6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:06.166212 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a087c38b-3fc0-48da-a73c-4194b2444ed6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:06.168422 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a7765-e428-4f49-9cf3-b126043d2ed8 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:06.182291 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 838/1661] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:06 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:19:06.183385 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a087c38b-3fc0-48da-a73c-4194b2444ed6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:06.184422 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bf14b2-02fe-48be-a04d-fcddd8c5e396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:06.195137 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf14b2-02fe-48be-a04d-fcddd8c5e396 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:06.196378 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf14b2-02fe-48be-a04d-fcddd8c5e396 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:06.207138 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a087c38b-3fc0-48da-a73c-4194b2444ed6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:06.207891 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf14b2-02fe-48be-a04d-fcddd8c5e396 admin admin] PATH_INFO: `/v3/users/a1744b7b411346038ff2a0b6dc95807d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:06.210705 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bf14b2-02fe-48be-a04d-fcddd8c5e396 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1744b7b411346038ff2a0b6dc95807d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:06.216052 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bf14b2-02fe-48be-a04d-fcddd8c5e396 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:06.219275 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7c3bc3-a212-4660-9908-8670bb5e0da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:06.219934 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7c3bc3-a212-4660-9908-8670bb5e0da2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:06.220223 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7c3bc3-a212-4660-9908-8670bb5e0da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:06.220510 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7c3bc3-a212-4660-9908-8670bb5e0da2 admin admin] PATH_INFO: `/v3/projects/7e55a370a92648a3a581bba404179ccc/users/bddd27afb6334099918ac9208fce25a8/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:06.264895 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7c3bc3-a212-4660-9908-8670bb5e0da2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e55a370a92648a3a581bba404179ccc, user_id=bddd27afb6334099918ac9208fce25a8, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:06.268018 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7c3bc3-a212-4660-9908-8670bb5e0da2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:06.297662 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 839/1662] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:19:06 2022] PUT /identity/v3/projects/7e55a370a92648a3a581bba404179ccc/users/bddd27afb6334099918ac9208fce25a8/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:06.300119 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7c3bc3-a212-4660-9908-8670bb5e0da2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:06.305924 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-da7c3bc3-a212-4660-9908-8670bb5e0da2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:06.316570 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f2bf14b2-02fe-48be-a04d-fcddd8c5e396 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1744b7b411346038ff2a0b6dc95807d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:06.333133 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f2bf14b2-02fe-48be-a04d-fcddd8c5e396 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1744b7b411346038ff2a0b6dc95807d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:06.334107 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f2bf14b2-02fe-48be-a04d-fcddd8c5e396 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1744b7b411346038ff2a0b6dc95807d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:06.354404 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f2bf14b2-02fe-48be-a04d-fcddd8c5e396 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1744b7b411346038ff2a0b6dc95807d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:06.383888 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f2bf14b2-02fe-48be-a04d-fcddd8c5e396 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1744b7b411346038ff2a0b6dc95807d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:06.393942 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 824/1663] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:19:06 2022] DELETE /identity/v3/users/a1744b7b411346038ff2a0b6dc95807d => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:06.400353 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0872941-2ba1-4d82-8994-b046b193cb34 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:06.400654 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0872941-2ba1-4d82-8994-b046b193cb34 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:06.400937 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0872941-2ba1-4d82-8994-b046b193cb34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:06.458055 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7c3bc3-a212-4660-9908-8670bb5e0da2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:06.465143 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-a0872941-2ba1-4d82-8994-b046b193cb34 None None] MFA Rules not processed for user `bddd27afb6334099918ac9208fce25a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:06.466413 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a0872941-2ba1-4d82-8994-b046b193cb34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:06.505317 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:06.581233 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6976bf2a-4650-4763-bd20-46a4749f5e96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:06.582885 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6976bf2a-4650-4763-bd20-46a4749f5e96 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:06.583403 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6976bf2a-4650-4763-bd20-46a4749f5e96 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:06.583805 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6976bf2a-4650-4763-bd20-46a4749f5e96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:06.586149 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6976bf2a-4650-4763-bd20-46a4749f5e96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:06.620625 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 825/1664] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:06.621290 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0872941-2ba1-4d82-8994-b046b193cb34 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:06.624211 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a0872941-2ba1-4d82-8994-b046b193cb34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:06.769281 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6976bf2a-4650-4763-bd20-46a4749f5e96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:06.773441 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6976bf2a-4650-4763-bd20-46a4749f5e96 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:06.808995 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 840/1665] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:19:06.813297 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0872941-2ba1-4d82-8994-b046b193cb34 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:06.820195 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf7331-2574-4e85-ba08-e975a5a00197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:06.821808 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf7331-2574-4e85-ba08-e975a5a00197 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:06.822311 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf7331-2574-4e85-ba08-e975a5a00197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:06.822672 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf7331-2574-4e85-ba08-e975a5a00197 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:06.824059 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaf7331-2574-4e85-ba08-e975a5a00197 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:06.826304 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaf7331-2574-4e85-ba08-e975a5a00197 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:06.888991 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 826/1666] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:19:06 2022] POST /identity/v3/projects => generated 381 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:19:06.889901 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6976bf2a-4650-4763-bd20-46a4749f5e96 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:06.904656 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6976bf2a-4650-4763-bd20-46a4749f5e96 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:06.939985 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:07.029514 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eabad70-c0bf-4555-a6d2-848f54e70d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:07.030507 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eabad70-c0bf-4555-a6d2-848f54e70d87 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:07.030947 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eabad70-c0bf-4555-a6d2-848f54e70d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:07.031237 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eabad70-c0bf-4555-a6d2-848f54e70d87 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:07.033180 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eabad70-c0bf-4555-a6d2-848f54e70d87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:07.036360 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eabad70-c0bf-4555-a6d2-848f54e70d87 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:07.073517 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 841/1667] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:19:06 2022] POST /identity/v3/users => generated 297 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:19:07.075525 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf7331-2574-4e85-ba08-e975a5a00197 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:07.095454 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf7331-2574-4e85-ba08-e975a5a00197 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:07.118773 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7f357e-7177-47d1-8af3-514f96e16c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:07.119522 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7f357e-7177-47d1-8af3-514f96e16c0f admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:07.119810 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7f357e-7177-47d1-8af3-514f96e16c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:07.120090 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7f357e-7177-47d1-8af3-514f96e16c0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:07.120853 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7f357e-7177-47d1-8af3-514f96e16c0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:07.122817 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7f357e-7177-47d1-8af3-514f96e16c0f admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:07.150880 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 827/1668] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:07 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:19:07.156298 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eabad70-c0bf-4555-a6d2-848f54e70d87 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:07.178373 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eabad70-c0bf-4555-a6d2-848f54e70d87 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:07.198594 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd2edde-1275-49ba-a725-5b86b5333e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:07.201106 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd2edde-1275-49ba-a725-5b86b5333e77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:07.201106 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd2edde-1275-49ba-a725-5b86b5333e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:07.201106 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd2edde-1275-49ba-a725-5b86b5333e77 admin admin] PATH_INFO: `/v3/projects/b18081aaec9c4dbd97844751037c7aa3/users/98fff7c679bd49938f97d4adde536f52/roles/f36e053b442540f08041ac7878760f65` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:07.231720 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd2edde-1275-49ba-a725-5b86b5333e77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b18081aaec9c4dbd97844751037c7aa3, user_id=98fff7c679bd49938f97d4adde536f52, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:07.235823 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd2edde-1275-49ba-a725-5b86b5333e77 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:07.276666 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 842/1669] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:19:07 2022] PUT /identity/v3/projects/b18081aaec9c4dbd97844751037c7aa3/users/98fff7c679bd49938f97d4adde536f52/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:07.277678 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7f357e-7177-47d1-8af3-514f96e16c0f admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:07.297101 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7f357e-7177-47d1-8af3-514f96e16c0f admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:07.413017 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3977a660-8517-4116-acdb-c5503d2f12a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:07.417380 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3977a660-8517-4116-acdb-c5503d2f12a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:07.417738 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3977a660-8517-4116-acdb-c5503d2f12a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:07.418093 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3977a660-8517-4116-acdb-c5503d2f12a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:07.418913 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3977a660-8517-4116-acdb-c5503d2f12a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:07.420669 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3977a660-8517-4116-acdb-c5503d2f12a7 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:07.456883 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 828/1670] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:07 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:19:07.460625 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd2edde-1275-49ba-a725-5b86b5333e77 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:07.485297 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd2edde-1275-49ba-a725-5b86b5333e77 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:07.490665 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d23d1-8cbd-4ed2-9cd9-300569b05349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:07.491849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d23d1-8cbd-4ed2-9cd9-300569b05349 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:07.492346 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d23d1-8cbd-4ed2-9cd9-300569b05349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:07.492850 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d23d1-8cbd-4ed2-9cd9-300569b05349 admin admin] PATH_INFO: `/v3/projects/b18081aaec9c4dbd97844751037c7aa3/users/98fff7c679bd49938f97d4adde536f52/roles/f36e053b442540f08041ac7878760f65` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:07.498526 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53d23d1-8cbd-4ed2-9cd9-300569b05349 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b18081aaec9c4dbd97844751037c7aa3, user_id=98fff7c679bd49938f97d4adde536f52, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:07.504003 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53d23d1-8cbd-4ed2-9cd9-300569b05349 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:07.528412 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 843/1671] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:19:07 2022] PUT /identity/v3/projects/b18081aaec9c4dbd97844751037c7aa3/users/98fff7c679bd49938f97d4adde536f52/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:19:07.541071 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f76e1-6fa6-48ed-99c2-de2da5cbdcab None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:07.545131 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f76e1-6fa6-48ed-99c2-de2da5cbdcab None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:07.545538 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f76e1-6fa6-48ed-99c2-de2da5cbdcab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:07.640386 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-493f76e1-6fa6-48ed-99c2-de2da5cbdcab None None] MFA Rules not processed for user `98fff7c679bd49938f97d4adde536f52`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:07.643382 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-493f76e1-6fa6-48ed-99c2-de2da5cbdcab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:07.714361 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:07.839587 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 829/1672] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:07 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:07.847796 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d23d1-8cbd-4ed2-9cd9-300569b05349 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:07.850054 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d53d23d1-8cbd-4ed2-9cd9-300569b05349 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:08.062284 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d23d1-8cbd-4ed2-9cd9-300569b05349 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:08.072375 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddf9b11-e746-41e2-980b-f595c9e08df8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:08.073198 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf9b11-e746-41e2-980b-f595c9e08df8 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:08.073563 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf9b11-e746-41e2-980b-f595c9e08df8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:08.074056 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf9b11-e746-41e2-980b-f595c9e08df8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:08.088171 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddf9b11-e746-41e2-980b-f595c9e08df8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:08.090689 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddf9b11-e746-41e2-980b-f595c9e08df8 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:08.105590 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 844/1673] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:07 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:19:08.334253 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493f76e1-6fa6-48ed-99c2-de2da5cbdcab None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:08.350421 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493f76e1-6fa6-48ed-99c2-de2da5cbdcab None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:08.359793 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfb9766-27b0-4c2c-a80b-fad3e500081f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:08.362276 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb9766-27b0-4c2c-a80b-fad3e500081f service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:08.363233 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb9766-27b0-4c2c-a80b-fad3e500081f service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:08.365195 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb9766-27b0-4c2c-a80b-fad3e500081f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:08.560646 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfb9766-27b0-4c2c-a80b-fad3e500081f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:08.565754 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfb9766-27b0-4c2c-a80b-fad3e500081f service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:08.591890 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 830/1674] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:08 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:19:09.144598 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddf9b11-e746-41e2-980b-f595c9e08df8 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:09.147008 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dddf9b11-e746-41e2-980b-f595c9e08df8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:09.256873 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfb9766-27b0-4c2c-a80b-fad3e500081f service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:09.262061 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-fbfb9766-27b0-4c2c-a80b-fad3e500081f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:09.324261 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddf9b11-e746-41e2-980b-f595c9e08df8 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:09.337062 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464af7a8-9624-41c1-8806-62bf55348e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:09.338245 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464af7a8-9624-41c1-8806-62bf55348e59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:09.338991 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464af7a8-9624-41c1-8806-62bf55348e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:09.339488 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464af7a8-9624-41c1-8806-62bf55348e59 admin admin] PATH_INFO: `/v3/projects/c31a26aa218c45aa88c4c7d3a737ecb0` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:09.342112 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464af7a8-9624-41c1-8806-62bf55348e59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c31a26aa218c45aa88c4c7d3a737ecb0)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:09.343420 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-464af7a8-9624-41c1-8806-62bf55348e59 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 09:19:09.343420 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Jul 04 09:19:09.343420 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: Jul 04 09:19:09.346616 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464af7a8-9624-41c1-8806-62bf55348e59 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:09.403809 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:09.529856 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-464af7a8-9624-41c1-8806-62bf55348e59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c31a26aa218c45aa88c4c7d3a737ecb0', 'request_id': 'req-464af7a8-9624-41c1-8806-62bf55348e59'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:09.541103 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfb9766-27b0-4c2c-a80b-fad3e500081f service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:09.547760 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-464af7a8-9624-41c1-8806-62bf55348e59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c31a26aa218c45aa88c4c7d3a737ecb0', 'request_id': 'req-464af7a8-9624-41c1-8806-62bf55348e59'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:09.581801 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-464af7a8-9624-41c1-8806-62bf55348e59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c31a26aa218c45aa88c4c7d3a737ecb0', 'request_id': 'req-464af7a8-9624-41c1-8806-62bf55348e59'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:09.611152 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-464af7a8-9624-41c1-8806-62bf55348e59 admin admin] The token cache is being invalidate because project c31a26aa218c45aa88c4c7d3a737ecb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:19:09.611332 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-464af7a8-9624-41c1-8806-62bf55348e59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:09.613417 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-464af7a8-9624-41c1-8806-62bf55348e59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:09.615614 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 845/1675] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:19:09 2022] DELETE /identity/v3/projects/c31a26aa218c45aa88c4c7d3a737ecb0 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:09.620421 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3772b59c-75aa-4fe5-bd40-5c0d08307b29 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:09.620713 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3772b59c-75aa-4fe5-bd40-5c0d08307b29 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:09.621007 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3772b59c-75aa-4fe5-bd40-5c0d08307b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:09.686311 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3772b59c-75aa-4fe5-bd40-5c0d08307b29 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:09.687591 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3772b59c-75aa-4fe5-bd40-5c0d08307b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:09.721743 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43d8948-7de4-47b7-a4c4-4c727c5124b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:09.722649 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d8948-7de4-47b7-a4c4-4c727c5124b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:09.724215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d8948-7de4-47b7-a4c4-4c727c5124b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:09.724711 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d8948-7de4-47b7-a4c4-4c727c5124b4 admin admin] PATH_INFO: `/v3/projects/6edc0208b3ca4ea49827030d358c315a` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:09.729315 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43d8948-7de4-47b7-a4c4-4c727c5124b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6edc0208b3ca4ea49827030d358c315a)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:09.737436 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43d8948-7de4-47b7-a4c4-4c727c5124b4 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:09.750412 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:09.843131 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 846/1676] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:09.851383 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e97acb-d58f-446a-b480-c0a4dce19dcd None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:09.852600 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e97acb-d58f-446a-b480-c0a4dce19dcd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:09.852898 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e97acb-d58f-446a-b480-c0a4dce19dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:09.893141 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e43d8948-7de4-47b7-a4c4-4c727c5124b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6edc0208b3ca4ea49827030d358c315a', 'request_id': 'req-e43d8948-7de4-47b7-a4c4-4c727c5124b4'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:09.909057 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-01e97acb-d58f-446a-b480-c0a4dce19dcd None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:09.910333 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-01e97acb-d58f-446a-b480-c0a4dce19dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:09.910559 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e43d8948-7de4-47b7-a4c4-4c727c5124b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6edc0208b3ca4ea49827030d358c315a', 'request_id': 'req-e43d8948-7de4-47b7-a4c4-4c727c5124b4'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:09.929268 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e43d8948-7de4-47b7-a4c4-4c727c5124b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6edc0208b3ca4ea49827030d358c315a', 'request_id': 'req-e43d8948-7de4-47b7-a4c4-4c727c5124b4'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:09.968070 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e43d8948-7de4-47b7-a4c4-4c727c5124b4 admin admin] The token cache is being invalidate because project 6edc0208b3ca4ea49827030d358c315a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:19:09.968623 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e43d8948-7de4-47b7-a4c4-4c727c5124b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:09.969394 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e43d8948-7de4-47b7-a4c4-4c727c5124b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:09.976557 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 831/1677] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:19:09 2022] DELETE /identity/v3/projects/6edc0208b3ca4ea49827030d358c315a => generated 0 bytes in 716 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:19:09.980792 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa274e01-917e-47ec-9ca9-449a6ac02672 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:09.982249 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa274e01-917e-47ec-9ca9-449a6ac02672 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:09.982540 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa274e01-917e-47ec-9ca9-449a6ac02672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:10.060958 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-fa274e01-917e-47ec-9ca9-449a6ac02672 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:10.062423 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-fa274e01-917e-47ec-9ca9-449a6ac02672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:10.169810 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 847/1678] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:10.172315 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e97acb-d58f-446a-b480-c0a4dce19dcd None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:10.223607 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e97acb-d58f-446a-b480-c0a4dce19dcd None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:10.252870 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bfdaaa-f7d3-4bf9-9852-6b26de3aed25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:10.260438 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bfdaaa-f7d3-4bf9-9852-6b26de3aed25 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:10.260733 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bfdaaa-f7d3-4bf9-9852-6b26de3aed25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:10.261180 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bfdaaa-f7d3-4bf9-9852-6b26de3aed25 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:10.265853 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bfdaaa-f7d3-4bf9-9852-6b26de3aed25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:19:10.269718 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bfdaaa-f7d3-4bf9-9852-6b26de3aed25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:10.279287 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bfdaaa-f7d3-4bf9-9852-6b26de3aed25 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:10.328619 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 848/1679] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:19:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:19:10.333593 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 832/1680] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:10.342727 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c405ce48-2533-4c2d-ab18-f1f33eb42ce6 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:10.343189 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c405ce48-2533-4c2d-ab18-f1f33eb42ce6 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:10.343589 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c405ce48-2533-4c2d-ab18-f1f33eb42ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:10.346076 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bfdaaa-f7d3-4bf9-9852-6b26de3aed25 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:10.379778 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bfdaaa-f7d3-4bf9-9852-6b26de3aed25 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:10.381432 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-c405ce48-2533-4c2d-ab18-f1f33eb42ce6 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:10.389337 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c405ce48-2533-4c2d-ab18-f1f33eb42ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:10.396023 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a560dc7a-5999-4815-a826-37150b849848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:10.397628 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a560dc7a-5999-4815-a826-37150b849848 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:10.398082 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a560dc7a-5999-4815-a826-37150b849848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:10.400347 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a560dc7a-5999-4815-a826-37150b849848 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:10.401257 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a560dc7a-5999-4815-a826-37150b849848 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:10.410048 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a560dc7a-5999-4815-a826-37150b849848 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:10.418151 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 833/1681] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:10.419767 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c405ce48-2533-4c2d-ab18-f1f33eb42ce6 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:10.443506 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c405ce48-2533-4c2d-ab18-f1f33eb42ce6 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:10.464885 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39af85a-168b-4289-b1d3-1216d10f0a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:10.466227 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39af85a-168b-4289-b1d3-1216d10f0a9d admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:10.466864 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39af85a-168b-4289-b1d3-1216d10f0a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:10.467469 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39af85a-168b-4289-b1d3-1216d10f0a9d admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:10.468839 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39af85a-168b-4289-b1d3-1216d10f0a9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:19:10.469531 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39af85a-168b-4289-b1d3-1216d10f0a9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:10.474282 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39af85a-168b-4289-b1d3-1216d10f0a9d admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:10.522217 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 849/1682] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:19:10 2022] POST /identity/v3/projects => generated 403 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:19:10.526291 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a560dc7a-5999-4815-a826-37150b849848 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:10.532133 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 834/1683] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:19:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:19:10.539101 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39af85a-168b-4289-b1d3-1216d10f0a9d admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:10.564268 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39af85a-168b-4289-b1d3-1216d10f0a9d admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:10.570014 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:10.732268 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f64eaa-1236-4eea-8aeb-00917bbc6da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:10.732971 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f64eaa-1236-4eea-8aeb-00917bbc6da6 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:10.733469 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f64eaa-1236-4eea-8aeb-00917bbc6da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:10.734301 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f64eaa-1236-4eea-8aeb-00917bbc6da6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:10.735866 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f64eaa-1236-4eea-8aeb-00917bbc6da6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:10.738735 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f64eaa-1236-4eea-8aeb-00917bbc6da6 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:10.777524 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a560dc7a-5999-4815-a826-37150b849848 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:10.784554 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:10.800623 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec78b6a5-340a-4652-8ef8-ca31328a3b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:10.804849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78b6a5-340a-4652-8ef8-ca31328a3b82 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:10.807354 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78b6a5-340a-4652-8ef8-ca31328a3b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:10.807667 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78b6a5-340a-4652-8ef8-ca31328a3b82 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:10.810503 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec78b6a5-340a-4652-8ef8-ca31328a3b82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:10.826195 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec78b6a5-340a-4652-8ef8-ca31328a3b82 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:10.835588 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 835/1684] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:19:10 2022] POST /identity/v3/projects => generated 409 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:19:10.837933 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f64eaa-1236-4eea-8aeb-00917bbc6da6 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:10.909550 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 850/1685] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:19:10 2022] POST /identity/v3/users => generated 302 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:19:10.916344 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec78b6a5-340a-4652-8ef8-ca31328a3b82 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:10.952321 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec78b6a5-340a-4652-8ef8-ca31328a3b82 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:11.011088 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f64eaa-1236-4eea-8aeb-00917bbc6da6 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:11.026119 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0f3282-7a9f-4390-b6b8-e993291c440a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:11.027874 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0f3282-7a9f-4390-b6b8-e993291c440a admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:11.030712 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0f3282-7a9f-4390-b6b8-e993291c440a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:11.032435 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0f3282-7a9f-4390-b6b8-e993291c440a admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:11.034980 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0f3282-7a9f-4390-b6b8-e993291c440a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:11.046998 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0f3282-7a9f-4390-b6b8-e993291c440a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:11.119917 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c69c21-7f53-4394-b90c-04e900be75b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:11.121087 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c69c21-7f53-4394-b90c-04e900be75b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:11.121692 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c69c21-7f53-4394-b90c-04e900be75b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:11.122146 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c69c21-7f53-4394-b90c-04e900be75b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:11.128254 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c69c21-7f53-4394-b90c-04e900be75b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:11.133782 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c69c21-7f53-4394-b90c-04e900be75b6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:11.156766 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 836/1686] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:19:10 2022] POST /identity/v3/users => generated 305 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:19:11.158401 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0f3282-7a9f-4390-b6b8-e993291c440a admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:11.162609 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 851/1687] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:10 2022] GET /identity/v3/roles => generated 1420 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:19:11.164007 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c69c21-7f53-4394-b90c-04e900be75b6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:11.188791 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c69c21-7f53-4394-b90c-04e900be75b6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:11.204397 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c108918-3f88-4089-930f-751a2e016359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:11.206105 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c108918-3f88-4089-930f-751a2e016359 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:11.207491 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c108918-3f88-4089-930f-751a2e016359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:11.208779 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c108918-3f88-4089-930f-751a2e016359 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:11.213057 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c108918-3f88-4089-930f-751a2e016359 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:11.218508 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c108918-3f88-4089-930f-751a2e016359 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:11.232373 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0f3282-7a9f-4390-b6b8-e993291c440a admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:11.256432 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 852/1688] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:19:11.261605 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c108918-3f88-4089-930f-751a2e016359 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:11.264963 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc774f6-4eeb-48db-9598-437acb70a5b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:11.267845 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc774f6-4eeb-48db-9598-437acb70a5b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:11.269712 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc774f6-4eeb-48db-9598-437acb70a5b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:11.273918 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc774f6-4eeb-48db-9598-437acb70a5b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:11.274943 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc774f6-4eeb-48db-9598-437acb70a5b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:11.282565 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc774f6-4eeb-48db-9598-437acb70a5b2 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:11.287617 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c108918-3f88-4089-930f-751a2e016359 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:11.295746 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdef48b-dbc1-4a9e-9e0c-da2f241f58e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:11.310741 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdef48b-dbc1-4a9e-9e0c-da2f241f58e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:11.311113 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdef48b-dbc1-4a9e-9e0c-da2f241f58e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:11.313031 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdef48b-dbc1-4a9e-9e0c-da2f241f58e8 admin admin] PATH_INFO: `/v3/projects/e3bf7211de5f483e8a55f4ca38246f2c/users/3ec0ef5374ad40e0831ac494b5ea5b11/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:11.344971 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 837/1689] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:19:11.356508 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc774f6-4eeb-48db-9598-437acb70a5b2 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:11.361747 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdef48b-dbc1-4a9e-9e0c-da2f241f58e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3bf7211de5f483e8a55f4ca38246f2c, user_id=3ec0ef5374ad40e0831ac494b5ea5b11, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:11.379577 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdef48b-dbc1-4a9e-9e0c-da2f241f58e8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:11.382598 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc774f6-4eeb-48db-9598-437acb70a5b2 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:11.402865 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1798c04b-d557-4422-8327-f32c71c1a17c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:11.403585 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798c04b-d557-4422-8327-f32c71c1a17c admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:11.403856 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798c04b-d557-4422-8327-f32c71c1a17c admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:11.404138 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798c04b-d557-4422-8327-f32c71c1a17c admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:11.405327 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1798c04b-d557-4422-8327-f32c71c1a17c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:11.407336 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1798c04b-d557-4422-8327-f32c71c1a17c admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:11.412907 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 853/1690] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:19:11 2022] PUT /identity/v3/projects/e3bf7211de5f483e8a55f4ca38246f2c/users/3ec0ef5374ad40e0831ac494b5ea5b11/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:11.419273 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4772faf-a9bd-441d-979b-f24dfff45d7f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:11.419673 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4772faf-a9bd-441d-979b-f24dfff45d7f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:11.420060 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4772faf-a9bd-441d-979b-f24dfff45d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:11.445326 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 838/1691] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:19:11.450421 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1798c04b-d557-4422-8327-f32c71c1a17c admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:11.476590 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1798c04b-d557-4422-8327-f32c71c1a17c admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:11.479910 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c4772faf-a9bd-441d-979b-f24dfff45d7f None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:11.481625 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c4772faf-a9bd-441d-979b-f24dfff45d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:11.660061 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629ad16c-ccf9-45d1-aa07-6544310e002c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:11.661177 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ad16c-ccf9-45d1-aa07-6544310e002c admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:11.661551 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ad16c-ccf9-45d1-aa07-6544310e002c admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:11.669385 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ad16c-ccf9-45d1-aa07-6544310e002c admin admin] PATH_INFO: `/v3/projects/d718667220734fec99f25f3850dfcbf5/users/79baf062f47a459bb5cbc12de5df5f60/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:11.682456 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 854/1692] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:11.714841 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629ad16c-ccf9-45d1-aa07-6544310e002c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d718667220734fec99f25f3850dfcbf5, user_id=79baf062f47a459bb5cbc12de5df5f60, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:11.715044 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4772faf-a9bd-441d-979b-f24dfff45d7f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:11.716523 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c4772faf-a9bd-441d-979b-f24dfff45d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:11.719022 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629ad16c-ccf9-45d1-aa07-6544310e002c admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:11.722228 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:11.760380 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 839/1693] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:19:11 2022] PUT /identity/v3/projects/d718667220734fec99f25f3850dfcbf5/users/79baf062f47a459bb5cbc12de5df5f60/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:11.764497 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ad34a5-55fd-4a44-a0e7-b8b0b489685e None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:11.764786 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ad34a5-55fd-4a44-a0e7-b8b0b489685e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:11.765113 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ad34a5-55fd-4a44-a0e7-b8b0b489685e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:11.897207 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-18ad34a5-55fd-4a44-a0e7-b8b0b489685e None None] MFA Rules not processed for user `79baf062f47a459bb5cbc12de5df5f60`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:11.899375 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-18ad34a5-55fd-4a44-a0e7-b8b0b489685e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:11.927444 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4772faf-a9bd-441d-979b-f24dfff45d7f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:12.024678 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:12.100735 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b9f08f-ae18-4695-8500-b22b5c02b082 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:12.101466 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b9f08f-ae18-4695-8500-b22b5c02b082 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:12.101807 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b9f08f-ae18-4695-8500-b22b5c02b082 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:12.102753 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b9f08f-ae18-4695-8500-b22b5c02b082 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:12.122192 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 840/1694] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:11 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:12.127271 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ad34a5-55fd-4a44-a0e7-b8b0b489685e None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:12.256114 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b9f08f-ae18-4695-8500-b22b5c02b082 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:12.257601 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b9f08f-ae18-4695-8500-b22b5c02b082 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:12.274572 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ad34a5-55fd-4a44-a0e7-b8b0b489685e None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:12.284978 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e25770a-702b-4895-836a-3a7ebe1725c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:12.285697 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e25770a-702b-4895-836a-3a7ebe1725c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:12.285987 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e25770a-702b-4895-836a-3a7ebe1725c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:12.286270 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e25770a-702b-4895-836a-3a7ebe1725c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:12.286979 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e25770a-702b-4895-836a-3a7ebe1725c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:12.288320 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 855/1695] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 575 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:19:12.291709 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fddedd-2491-4fa1-9c98-5813285ab985 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:12.292000 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fddedd-2491-4fa1-9c98-5813285ab985 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:12.292284 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fddedd-2491-4fa1-9c98-5813285ab985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:12.296979 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e25770a-702b-4895-836a-3a7ebe1725c8 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:12.397716 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 841/1696] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:19:12 2022] POST /identity/v3/projects => generated 405 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:19:12.401276 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e25770a-702b-4895-836a-3a7ebe1725c8 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:12.408339 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b2fddedd-2491-4fa1-9c98-5813285ab985 None None] MFA Rules not processed for user `6bc30b7d536444008e5b5bab45c5c438`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:12.409949 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b2fddedd-2491-4fa1-9c98-5813285ab985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:12.567826 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e25770a-702b-4895-836a-3a7ebe1725c8 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:12.587504 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3034d8b-f46e-443f-8fc4-62911eb4107c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:12.588825 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3034d8b-f46e-443f-8fc4-62911eb4107c admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:12.589259 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3034d8b-f46e-443f-8fc4-62911eb4107c admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:12.589643 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3034d8b-f46e-443f-8fc4-62911eb4107c admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:12.590454 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3034d8b-f46e-443f-8fc4-62911eb4107c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:12.599238 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3034d8b-f46e-443f-8fc4-62911eb4107c admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:12.625995 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:12.646694 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 856/1697] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:12.655386 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fddedd-2491-4fa1-9c98-5813285ab985 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:12.656458 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 842/1698] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:19:12 2022] POST /identity/v3/users => generated 309 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:19:12.659828 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b2fddedd-2491-4fa1-9c98-5813285ab985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:12.660202 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3034d8b-f46e-443f-8fc4-62911eb4107c admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:12.688032 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3034d8b-f46e-443f-8fc4-62911eb4107c admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:12.701807 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e698ca6b-b24e-43a0-a61d-05d4ae6dfccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:12.702876 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e698ca6b-b24e-43a0-a61d-05d4ae6dfccd admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:12.704573 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e698ca6b-b24e-43a0-a61d-05d4ae6dfccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:12.705157 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e698ca6b-b24e-43a0-a61d-05d4ae6dfccd admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:12.706435 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e698ca6b-b24e-43a0-a61d-05d4ae6dfccd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:12.708444 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e698ca6b-b24e-43a0-a61d-05d4ae6dfccd admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:12.751121 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 843/1699] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:19:12.752516 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e698ca6b-b24e-43a0-a61d-05d4ae6dfccd admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:12.775557 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e698ca6b-b24e-43a0-a61d-05d4ae6dfccd admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:12.796375 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19238f52-23a2-4230-9301-8233583008d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:12.798123 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19238f52-23a2-4230-9301-8233583008d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:12.798614 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19238f52-23a2-4230-9301-8233583008d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:12.799099 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19238f52-23a2-4230-9301-8233583008d1 admin admin] PATH_INFO: `/v3/projects/8d3a17536478469e80e0e431e9fa8b17/users/e3035d47ea4f4fe8af7107fc5ffa63bb/roles/f36e053b442540f08041ac7878760f65` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:12.842684 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19238f52-23a2-4230-9301-8233583008d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d3a17536478469e80e0e431e9fa8b17, user_id=e3035d47ea4f4fe8af7107fc5ffa63bb, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:12.846243 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19238f52-23a2-4230-9301-8233583008d1 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:12.872172 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 844/1700] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:19:12 2022] PUT /identity/v3/projects/8d3a17536478469e80e0e431e9fa8b17/users/e3035d47ea4f4fe8af7107fc5ffa63bb/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:12.876424 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19238f52-23a2-4230-9301-8233583008d1 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:12.913505 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:12.975664 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fddedd-2491-4fa1-9c98-5813285ab985 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:12.984049 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19238f52-23a2-4230-9301-8233583008d1 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:13.002930 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ba621d-9581-444b-853c-fcaf5da15d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:13.003865 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba621d-9581-444b-853c-fcaf5da15d04 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:13.004253 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba621d-9581-444b-853c-fcaf5da15d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:13.004676 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba621d-9581-444b-853c-fcaf5da15d04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:13.006243 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ba621d-9581-444b-853c-fcaf5da15d04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:13.009065 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ba621d-9581-444b-853c-fcaf5da15d04 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:13.026971 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 845/1701] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:19:13.032472 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ba621d-9581-444b-853c-fcaf5da15d04 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:13.067012 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ba621d-9581-444b-853c-fcaf5da15d04 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:13.073905 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48274173-9d6b-40de-b7d5-dce6a283caf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:13.075060 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48274173-9d6b-40de-b7d5-dce6a283caf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:13.075451 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48274173-9d6b-40de-b7d5-dce6a283caf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:13.075832 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48274173-9d6b-40de-b7d5-dce6a283caf0 admin admin] PATH_INFO: `/v3/projects/8d3a17536478469e80e0e431e9fa8b17/users/e3035d47ea4f4fe8af7107fc5ffa63bb/roles/f36e053b442540f08041ac7878760f65` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:13.078257 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48274173-9d6b-40de-b7d5-dce6a283caf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d3a17536478469e80e0e431e9fa8b17, user_id=e3035d47ea4f4fe8af7107fc5ffa63bb, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:13.081994 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48274173-9d6b-40de-b7d5-dce6a283caf0 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:13.105079 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 846/1702] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:19:13 2022] PUT /identity/v3/projects/8d3a17536478469e80e0e431e9fa8b17/users/e3035d47ea4f4fe8af7107fc5ffa63bb/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:13.118050 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1471c9-3c62-456d-a933-828621582743 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:13.119607 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1471c9-3c62-456d-a933-828621582743 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:13.120248 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1471c9-3c62-456d-a933-828621582743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:13.179381 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-cd1471c9-3c62-456d-a933-828621582743 None None] MFA Rules not processed for user `e3035d47ea4f4fe8af7107fc5ffa63bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:13.181314 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-cd1471c9-3c62-456d-a933-828621582743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:13.223988 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faba687-9bbd-4563-8897-aff6a297b055 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15039da209ec445d93d49e2bd6d854eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:13.224585 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faba687-9bbd-4563-8897-aff6a297b055 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:13.224871 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faba687-9bbd-4563-8897-aff6a297b055 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:13.225178 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faba687-9bbd-4563-8897-aff6a297b055 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:13.347602 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 847/1703] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:13 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:13.419896 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1471c9-3c62-456d-a933-828621582743 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:13.421597 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-cd1471c9-3c62-456d-a933-828621582743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:13.436258 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faba687-9bbd-4563-8897-aff6a297b055 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:13.444200 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faba687-9bbd-4563-8897-aff6a297b055 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:13.471790 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 857/1704] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 817 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:19:13.580284 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1471c9-3c62-456d-a933-828621582743 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:13.593974 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df67f4ac-63db-4d6d-afda-63884458ee8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:13.594653 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df67f4ac-63db-4d6d-afda-63884458ee8a service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:13.594940 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df67f4ac-63db-4d6d-afda-63884458ee8a service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:13.595222 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df67f4ac-63db-4d6d-afda-63884458ee8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:13.625816 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df67f4ac-63db-4d6d-afda-63884458ee8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:13.628529 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df67f4ac-63db-4d6d-afda-63884458ee8a service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:13.672037 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 848/1705] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:13 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:19:14.022597 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faba687-9bbd-4563-8897-aff6a297b055 service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:14.113421 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:14.170326 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faba687-9bbd-4563-8897-aff6a297b055 service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:14.175992 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6f6063-9b27-482f-8f63-2c81ab97ef3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:14.176673 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f6063-9b27-482f-8f63-2c81ab97ef3a service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:14.176958 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f6063-9b27-482f-8f63-2c81ab97ef3a service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:14.177281 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f6063-9b27-482f-8f63-2c81ab97ef3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:14.302778 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6f6063-9b27-482f-8f63-2c81ab97ef3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:14.304477 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6f6063-9b27-482f-8f63-2c81ab97ef3a service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:14.321298 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 858/1706] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:14 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:19:16.632142 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa99f840-f1da-4342-a2bd-d9418ddf3c67 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:16.632691 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa99f840-f1da-4342-a2bd-d9418ddf3c67 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:16.632775 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa99f840-f1da-4342-a2bd-d9418ddf3c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:16.713262 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-fa99f840-f1da-4342-a2bd-d9418ddf3c67 None None] MFA Rules not processed for user `60d06d081a29460b8e85b0b301399f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:16.723389 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-fa99f840-f1da-4342-a2bd-d9418ddf3c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:16.777093 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:16.875539 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 849/1707] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:16 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:16.891031 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6f6063-9b27-482f-8f63-2c81ab97ef3a service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:16.893101 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2a6f6063-9b27-482f-8f63-2c81ab97ef3a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:17.072216 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6f6063-9b27-482f-8f63-2c81ab97ef3a service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:17.085878 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511e8172-1c31-455a-a3d5-abd929ed67fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15039da209ec445d93d49e2bd6d854eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:17.090125 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e8172-1c31-455a-a3d5-abd929ed67fe service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:17.090581 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e8172-1c31-455a-a3d5-abd929ed67fe service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:17.092837 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e8172-1c31-455a-a3d5-abd929ed67fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:17.108753 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511e8172-1c31-455a-a3d5-abd929ed67fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:17.110903 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511e8172-1c31-455a-a3d5-abd929ed67fe service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:17.146167 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 859/1708] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:16 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:19:18.593923 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f827170e-3bd4-4554-9b93-673a15ba7cb1 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:18.594731 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f827170e-3bd4-4554-9b93-673a15ba7cb1 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:18.595159 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f827170e-3bd4-4554-9b93-673a15ba7cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:18.659026 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-f827170e-3bd4-4554-9b93-673a15ba7cb1 None None] MFA Rules not processed for user `3ec0ef5374ad40e0831ac494b5ea5b11`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:18.660723 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f827170e-3bd4-4554-9b93-673a15ba7cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:18.818368 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 850/1709] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:18 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:18.818457 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511e8172-1c31-455a-a3d5-abd929ed67fe service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:18.927659 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511e8172-1c31-455a-a3d5-abd929ed67fe service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:18.934160 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6efe761-d4a9-4485-a85a-db8aec02273a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:18.935019 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6efe761-d4a9-4485-a85a-db8aec02273a admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:18.935376 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6efe761-d4a9-4485-a85a-db8aec02273a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:18.935723 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6efe761-d4a9-4485-a85a-db8aec02273a admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:18.936637 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6efe761-d4a9-4485-a85a-db8aec02273a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:18.939629 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6efe761-d4a9-4485-a85a-db8aec02273a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:18.969790 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:18.994187 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f827170e-3bd4-4554-9b93-673a15ba7cb1 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:19.003381 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 860/1710] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:19:18 2022] POST /identity/v3/projects => generated 403 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:19:19.012224 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f827170e-3bd4-4554-9b93-673a15ba7cb1 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:19.163636 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dee9ef-a576-4587-913e-79928267c5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:19.166877 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dee9ef-a576-4587-913e-79928267c5dd admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:19.167482 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dee9ef-a576-4587-913e-79928267c5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:19.167482 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dee9ef-a576-4587-913e-79928267c5dd admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:19.168162 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dee9ef-a576-4587-913e-79928267c5dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:19.171012 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dee9ef-a576-4587-913e-79928267c5dd admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:19.200535 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 851/1711] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:19:18 2022] POST /identity/v3/users => generated 308 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:19:19.204178 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6efe761-d4a9-4485-a85a-db8aec02273a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:19.217403 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6efe761-d4a9-4485-a85a-db8aec02273a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:19.223109 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cead58-87d5-4d4c-b901-2d8b3eacecc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:19.224399 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cead58-87d5-4d4c-b901-2d8b3eacecc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:19.224897 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cead58-87d5-4d4c-b901-2d8b3eacecc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:19.225668 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cead58-87d5-4d4c-b901-2d8b3eacecc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:19.226704 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cead58-87d5-4d4c-b901-2d8b3eacecc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:19.230052 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cead58-87d5-4d4c-b901-2d8b3eacecc9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:19.251223 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 861/1712] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:19 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:19:19.252042 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dee9ef-a576-4587-913e-79928267c5dd admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:19.269806 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dee9ef-a576-4587-913e-79928267c5dd admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:19.281755 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3faf65-0e10-4adc-8940-6d6b25b01687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:19.283191 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3faf65-0e10-4adc-8940-6d6b25b01687 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:19.283507 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3faf65-0e10-4adc-8940-6d6b25b01687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:19.283792 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3faf65-0e10-4adc-8940-6d6b25b01687 admin admin] PATH_INFO: `/v3/projects/d93ad51f26d04cf3ba554fd45dc7116f/users/ed486d69757b4f50996af383aeecd90b/roles/f36e053b442540f08041ac7878760f65` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:19.304576 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3faf65-0e10-4adc-8940-6d6b25b01687 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d93ad51f26d04cf3ba554fd45dc7116f, user_id=ed486d69757b4f50996af383aeecd90b, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:19.309498 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3faf65-0e10-4adc-8940-6d6b25b01687 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:19.332602 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 852/1713] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:19:19 2022] PUT /identity/v3/projects/d93ad51f26d04cf3ba554fd45dc7116f/users/ed486d69757b4f50996af383aeecd90b/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:19.334836 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cead58-87d5-4d4c-b901-2d8b3eacecc9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:19.353518 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cead58-87d5-4d4c-b901-2d8b3eacecc9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:19.453897 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1090ecf-7ab7-4d91-9afa-52c43a672d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:19.454786 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1090ecf-7ab7-4d91-9afa-52c43a672d33 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:19.455078 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1090ecf-7ab7-4d91-9afa-52c43a672d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:19.455458 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1090ecf-7ab7-4d91-9afa-52c43a672d33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:19.456347 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1090ecf-7ab7-4d91-9afa-52c43a672d33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:19.458593 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1090ecf-7ab7-4d91-9afa-52c43a672d33 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:19.472604 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 862/1714] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:19 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:19:19.478115 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3faf65-0e10-4adc-8940-6d6b25b01687 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:19.496746 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3faf65-0e10-4adc-8940-6d6b25b01687 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:19.503616 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1cb776-4cfc-485a-b781-d73d82c12b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:19.505753 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1cb776-4cfc-485a-b781-d73d82c12b51 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:19.506953 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1cb776-4cfc-485a-b781-d73d82c12b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:19.507684 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1cb776-4cfc-485a-b781-d73d82c12b51 admin admin] PATH_INFO: `/v3/projects/d93ad51f26d04cf3ba554fd45dc7116f/users/ed486d69757b4f50996af383aeecd90b/roles/f36e053b442540f08041ac7878760f65` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:19.515187 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1cb776-4cfc-485a-b781-d73d82c12b51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d93ad51f26d04cf3ba554fd45dc7116f, user_id=ed486d69757b4f50996af383aeecd90b, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:19.522610 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1cb776-4cfc-485a-b781-d73d82c12b51 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:19.537872 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:19.557199 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 853/1715] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:19:19 2022] PUT /identity/v3/projects/d93ad51f26d04cf3ba554fd45dc7116f/users/ed486d69757b4f50996af383aeecd90b/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:19:19.943109 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb84d57-0aef-461c-a0c8-ff8cd95e5863 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:19.943564 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb84d57-0aef-461c-a0c8-ff8cd95e5863 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:19.943671 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb84d57-0aef-461c-a0c8-ff8cd95e5863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:20.004508 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9cb84d57-0aef-461c-a0c8-ff8cd95e5863 None None] MFA Rules not processed for user `e50f9b53a10a43fa9b5fec03d01f451f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:20.006193 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9cb84d57-0aef-461c-a0c8-ff8cd95e5863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:20.124027 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 863/1716] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:20.139431 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1cb776-4cfc-485a-b781-d73d82c12b51 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:20.427725 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1cb776-4cfc-485a-b781-d73d82c12b51 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:20.449012 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107a010b-5dc6-4a4d-8424-2f5e93464534 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15039da209ec445d93d49e2bd6d854eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:20.449952 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a010b-5dc6-4a4d-8424-2f5e93464534 service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:20.450598 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a010b-5dc6-4a4d-8424-2f5e93464534 service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:20.451029 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a010b-5dc6-4a4d-8424-2f5e93464534 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:20.462306 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107a010b-5dc6-4a4d-8424-2f5e93464534 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:20.481472 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107a010b-5dc6-4a4d-8424-2f5e93464534 service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:20.506288 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 854/1717] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:20 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:19:20.978468 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc9678c-8f7c-4b5c-9def-77f2e779d51c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:20.978948 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc9678c-8f7c-4b5c-9def-77f2e779d51c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:20.978948 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc9678c-8f7c-4b5c-9def-77f2e779d51c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:21.034929 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-bfc9678c-8f7c-4b5c-9def-77f2e779d51c None None] MFA Rules not processed for user `60d06d081a29460b8e85b0b301399f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:21.039124 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bfc9678c-8f7c-4b5c-9def-77f2e779d51c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:21.183204 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 864/1718] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:21.625996 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4419bbd5-5e40-4cfc-9e9c-3d252858125f None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:21.626951 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4419bbd5-5e40-4cfc-9e9c-3d252858125f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:21.627377 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4419bbd5-5e40-4cfc-9e9c-3d252858125f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:21.671714 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-4419bbd5-5e40-4cfc-9e9c-3d252858125f None None] MFA Rules not processed for user `6bc30b7d536444008e5b5bab45c5c438`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:21.676761 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4419bbd5-5e40-4cfc-9e9c-3d252858125f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:21.807984 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 855/1719] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:22.105827 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077dfcc6-1f96-497a-99da-476910ece759 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:22.106284 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077dfcc6-1f96-497a-99da-476910ece759 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:22.106373 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077dfcc6-1f96-497a-99da-476910ece759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:22.170640 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-077dfcc6-1f96-497a-99da-476910ece759 None None] MFA Rules not processed for user `ed486d69757b4f50996af383aeecd90b`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:22.178855 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-077dfcc6-1f96-497a-99da-476910ece759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:22.194539 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:22.269330 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd7c9e7-166e-425e-87aa-2cfe44ee6999 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:22.269629 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd7c9e7-166e-425e-87aa-2cfe44ee6999 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:22.269918 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd7c9e7-166e-425e-87aa-2cfe44ee6999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:22.384827 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-0cd7c9e7-166e-425e-87aa-2cfe44ee6999 None None] MFA Rules not processed for user `e034114ab8244cc99dbf977b458a43f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:22.388215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0cd7c9e7-166e-425e-87aa-2cfe44ee6999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:22.411195 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 865/1720] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:22 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:22.415676 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b187946-d817-4e51-9d31-0fbcda168994 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:22.420409 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b187946-d817-4e51-9d31-0fbcda168994 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:22.420706 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b187946-d817-4e51-9d31-0fbcda168994 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:22.422348 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 866/1721] 10.222.0.71 () {54 vars in 912 bytes} [Mon Jul 4 09:19:22 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:19:22.434203 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925f8815-a515-465e-9207-4ca2a0323345 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:22.435069 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925f8815-a515-465e-9207-4ca2a0323345 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:22.435894 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925f8815-a515-465e-9207-4ca2a0323345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:22.542733 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-925f8815-a515-465e-9207-4ca2a0323345 None None] MFA Rules not processed for user `15818e58661646f381da0973d4c2ff64`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:22.549994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-925f8815-a515-465e-9207-4ca2a0323345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:22.626106 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 856/1722] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:22.637471 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfd646c-fdd5-4785-9b3e-5a1da459190c None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:22.638117 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfd646c-fdd5-4785-9b3e-5a1da459190c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:22.638700 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfd646c-fdd5-4785-9b3e-5a1da459190c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:22.702824 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-3bfd646c-fdd5-4785-9b3e-5a1da459190c None None] MFA Rules not processed for user `e034114ab8244cc99dbf977b458a43f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:22.705734 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-3bfd646c-fdd5-4785-9b3e-5a1da459190c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:22.736368 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 857/1723] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:19:22.740115 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 867/1724] 10.222.0.71 () {58 vars in 1009 bytes} [Mon Jul 4 09:19:22 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:22.928715 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfd646c-fdd5-4785-9b3e-5a1da459190c None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:22.947239 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfd646c-fdd5-4785-9b3e-5a1da459190c None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:22.960586 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7905af07-83c7-4252-a2c1-d3a57513596e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:22.962803 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7905af07-83c7-4252-a2c1-d3a57513596e service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:22.964069 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7905af07-83c7-4252-a2c1-d3a57513596e service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:22.964822 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7905af07-83c7-4252-a2c1-d3a57513596e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:22.990951 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7905af07-83c7-4252-a2c1-d3a57513596e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:22.996111 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7905af07-83c7-4252-a2c1-d3a57513596e service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:23.006802 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:23.023142 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 858/1725] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:19:23.344783 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b62b0c-b055-4dd3-a110-632b832ea271 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:23.345967 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b62b0c-b055-4dd3-a110-632b832ea271 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:23.346435 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b62b0c-b055-4dd3-a110-632b832ea271 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:23.348857 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 868/1726] 10.222.0.71 () {54 vars in 912 bytes} [Mon Jul 4 09:19:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:19:23.364675 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca9daa-381d-4404-9819-4840b94fad7f None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:23.365243 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca9daa-381d-4404-9819-4840b94fad7f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:23.365669 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca9daa-381d-4404-9819-4840b94fad7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:23.425999 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-00ca9daa-381d-4404-9819-4840b94fad7f None None] MFA Rules not processed for user `15818e58661646f381da0973d4c2ff64`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:23.426825 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-00ca9daa-381d-4404-9819-4840b94fad7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:23.448567 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 859/1727] 10.222.0.71 () {58 vars in 1009 bytes} [Mon Jul 4 09:19:23 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:23.514202 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b62b0c-b055-4dd3-a110-632b832ea271 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:23.531732 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b62b0c-b055-4dd3-a110-632b832ea271 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:23.543890 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78de63bf-1e01-47bb-aef6-71f5cfa13677 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:23.545626 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de63bf-1e01-47bb-aef6-71f5cfa13677 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:23.546613 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de63bf-1e01-47bb-aef6-71f5cfa13677 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:23.547333 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de63bf-1e01-47bb-aef6-71f5cfa13677 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:23.569449 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78de63bf-1e01-47bb-aef6-71f5cfa13677 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:23.574692 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78de63bf-1e01-47bb-aef6-71f5cfa13677 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:23.600122 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 869/1728] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:19:24.043446 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ca9daa-381d-4404-9819-4840b94fad7f None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:24.073780 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ca9daa-381d-4404-9819-4840b94fad7f None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:24.111521 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432c1883-ad7b-4ef2-86dc-199bd0aa2935 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15039da209ec445d93d49e2bd6d854eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:24.112186 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c1883-ad7b-4ef2-86dc-199bd0aa2935 service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:24.112616 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c1883-ad7b-4ef2-86dc-199bd0aa2935 service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:24.112947 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c1883-ad7b-4ef2-86dc-199bd0aa2935 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:24.156092 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432c1883-ad7b-4ef2-86dc-199bd0aa2935 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:24.164768 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432c1883-ad7b-4ef2-86dc-199bd0aa2935 service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:24.183582 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 860/1729] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:24 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:19:24.219341 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687611a6-a842-47c6-9224-75ca1fab2f57 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:24.220130 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687611a6-a842-47c6-9224-75ca1fab2f57 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:24.220548 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687611a6-a842-47c6-9224-75ca1fab2f57 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:24.222583 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 870/1730] 10.222.0.71 () {54 vars in 960 bytes} [Mon Jul 4 09:19:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:19:24.229274 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545dcd8-4e1a-48cd-ae8d-e22ed74a6214 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:24.230021 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545dcd8-4e1a-48cd-ae8d-e22ed74a6214 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:24.230514 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545dcd8-4e1a-48cd-ae8d-e22ed74a6214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:24.233580 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c0e41-f111-4d9b-92b1-a6f40f0e6901 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:24.233868 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c0e41-f111-4d9b-92b1-a6f40f0e6901 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:24.234152 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c0e41-f111-4d9b-92b1-a6f40f0e6901 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:24.236175 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 871/1731] 10.222.0.71 () {54 vars in 921 bytes} [Mon Jul 4 09:19:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:19:24.242704 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4ebce-eea2-4b3e-ac93-d2c4f1a26017 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:24.243239 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4ebce-eea2-4b3e-ac93-d2c4f1a26017 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:24.243662 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4ebce-eea2-4b3e-ac93-d2c4f1a26017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:24.294051 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-0545dcd8-4e1a-48cd-ae8d-e22ed74a6214 None None] MFA Rules not processed for user `293c695f15c8437d81a619f840318f44`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:24.299648 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0545dcd8-4e1a-48cd-ae8d-e22ed74a6214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:24.313270 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f9f4ebce-eea2-4b3e-ac93-d2c4f1a26017 None None] MFA Rules not processed for user `293c695f15c8437d81a619f840318f44`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:24.317965 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f9f4ebce-eea2-4b3e-ac93-d2c4f1a26017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:24.577484 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 861/1732] 10.222.0.71 () {58 vars in 1057 bytes} [Mon Jul 4 09:19:24 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:24.590545 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 872/1733] 10.222.0.71 () {58 vars in 1018 bytes} [Mon Jul 4 09:19:24 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:24.597487 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0545dcd8-4e1a-48cd-ae8d-e22ed74a6214 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:24.602189 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f4ebce-eea2-4b3e-ac93-d2c4f1a26017 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:24.603823 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f9f4ebce-eea2-4b3e-ac93-d2c4f1a26017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:24.631851 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0545dcd8-4e1a-48cd-ae8d-e22ed74a6214 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:24.642268 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f4ebce-eea2-4b3e-ac93-d2c4f1a26017 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:24.662728 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0856c48b-d395-434a-af1b-e97ffff4106a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '293c695f15c8437d81a619f840318f44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:24.663667 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0856c48b-d395-434a-af1b-e97ffff4106a service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:24.664310 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0856c48b-d395-434a-af1b-e97ffff4106a service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:24.664565 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62492f9-3cf3-4555-80a3-b1402b39b513 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '293c695f15c8437d81a619f840318f44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:24.665064 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0856c48b-d395-434a-af1b-e97ffff4106a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:24.670636 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62492f9-3cf3-4555-80a3-b1402b39b513 service placement] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:24.677084 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62492f9-3cf3-4555-80a3-b1402b39b513 service placement] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:24.677567 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62492f9-3cf3-4555-80a3-b1402b39b513 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:24.681408 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c62492f9-3cf3-4555-80a3-b1402b39b513 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:24.682585 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0856c48b-d395-434a-af1b-e97ffff4106a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:24.684513 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0856c48b-d395-434a-af1b-e97ffff4106a service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:24.716104 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 873/1734] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:19:24.737887 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62492f9-3cf3-4555-80a3-b1402b39b513 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:24.739637 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62492f9-3cf3-4555-80a3-b1402b39b513 service placement] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:24.774477 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 862/1735] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:19:24.807720 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0856c48b-d395-434a-af1b-e97ffff4106a service placement] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:24.810887 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0856c48b-d395-434a-af1b-e97ffff4106a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:24.839268 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:24.958299 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0856c48b-d395-434a-af1b-e97ffff4106a service placement] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:24.975389 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc05b00c-64ef-4a91-8620-08b499e42e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:24.976857 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc05b00c-64ef-4a91-8620-08b499e42e83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:24.977393 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc05b00c-64ef-4a91-8620-08b499e42e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:24.977715 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc05b00c-64ef-4a91-8620-08b499e42e83 admin admin] PATH_INFO: `/v3/users/bddd27afb6334099918ac9208fce25a8` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:24.981372 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc05b00c-64ef-4a91-8620-08b499e42e83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bddd27afb6334099918ac9208fce25a8)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:24.995151 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc05b00c-64ef-4a91-8620-08b499e42e83 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:25.139889 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62492f9-3cf3-4555-80a3-b1402b39b513 service placement] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:25.141412 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c62492f9-3cf3-4555-80a3-b1402b39b513 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:25.144565 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dc05b00c-64ef-4a91-8620-08b499e42e83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bddd27afb6334099918ac9208fce25a8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:25.145607 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dc05b00c-64ef-4a91-8620-08b499e42e83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bddd27afb6334099918ac9208fce25a8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:25.159997 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dc05b00c-64ef-4a91-8620-08b499e42e83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bddd27afb6334099918ac9208fce25a8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:25.175262 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62492f9-3cf3-4555-80a3-b1402b39b513 service placement] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:25.194030 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281d7b32-f60d-4da9-88ae-550b7ee670f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:25.196028 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d7b32-f60d-4da9-88ae-550b7ee670f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:25.196637 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d7b32-f60d-4da9-88ae-550b7ee670f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:25.197147 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d7b32-f60d-4da9-88ae-550b7ee670f6 admin admin] PATH_INFO: `/v3/users/983b3cf7bb4d4eb29e7cef371423db20` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:25.200431 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281d7b32-f60d-4da9-88ae-550b7ee670f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=983b3cf7bb4d4eb29e7cef371423db20)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:25.202347 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dc05b00c-64ef-4a91-8620-08b499e42e83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bddd27afb6334099918ac9208fce25a8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:25.203239 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dc05b00c-64ef-4a91-8620-08b499e42e83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bddd27afb6334099918ac9208fce25a8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:25.207981 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281d7b32-f60d-4da9-88ae-550b7ee670f6 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:25.236306 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 874/1736] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:19:24 2022] DELETE /identity/v3/users/bddd27afb6334099918ac9208fce25a8 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:25.242947 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca285685-6409-47dd-8535-fc10900d016f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:25.243102 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca285685-6409-47dd-8535-fc10900d016f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:25.243885 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca285685-6409-47dd-8535-fc10900d016f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:25.281749 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ca285685-6409-47dd-8535-fc10900d016f None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:25.285906 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ca285685-6409-47dd-8535-fc10900d016f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:25.324103 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-281d7b32-f60d-4da9-88ae-550b7ee670f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '983b3cf7bb4d4eb29e7cef371423db20'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:25.343495 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-281d7b32-f60d-4da9-88ae-550b7ee670f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '983b3cf7bb4d4eb29e7cef371423db20'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:25.345482 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-281d7b32-f60d-4da9-88ae-550b7ee670f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '983b3cf7bb4d4eb29e7cef371423db20'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:25.363106 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-281d7b32-f60d-4da9-88ae-550b7ee670f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '983b3cf7bb4d4eb29e7cef371423db20'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:25.418283 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-281d7b32-f60d-4da9-88ae-550b7ee670f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '983b3cf7bb4d4eb29e7cef371423db20'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:25.436816 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 863/1737] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:19:25 2022] DELETE /identity/v3/users/983b3cf7bb4d4eb29e7cef371423db20 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:19:25.441784 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d8c98-ce9b-4203-b7f1-ae4a19563c59 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:25.442089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d8c98-ce9b-4203-b7f1-ae4a19563c59 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:25.442493 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d8c98-ce9b-4203-b7f1-ae4a19563c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:25.460172 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 875/1738] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:25.466872 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca285685-6409-47dd-8535-fc10900d016f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:25.474732 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ca285685-6409-47dd-8535-fc10900d016f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:25.485977 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-b01d8c98-ce9b-4203-b7f1-ae4a19563c59 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:25.488401 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b01d8c98-ce9b-4203-b7f1-ae4a19563c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:25.525759 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:25.566568 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca285685-6409-47dd-8535-fc10900d016f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:25.592322 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:25.651794 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 864/1739] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:25.654019 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69d6aee-a487-4ad3-9d9f-a98da1b0977f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:25.655107 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69d6aee-a487-4ad3-9d9f-a98da1b0977f service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:25.655526 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69d6aee-a487-4ad3-9d9f-a98da1b0977f service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:25.655921 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69d6aee-a487-4ad3-9d9f-a98da1b0977f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:25.663332 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01d8c98-ce9b-4203-b7f1-ae4a19563c59 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:25.669882 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69d6aee-a487-4ad3-9d9f-a98da1b0977f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:25.672325 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69d6aee-a487-4ad3-9d9f-a98da1b0977f service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:25.678995 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01d8c98-ce9b-4203-b7f1-ae4a19563c59 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:25.686821 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 876/1740] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:19:25.691816 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5b2160-ed9f-480e-ba91-756407fa1b9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:25.692629 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5b2160-ed9f-480e-ba91-756407fa1b9b service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:25.693067 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5b2160-ed9f-480e-ba91-756407fa1b9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:25.693634 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5b2160-ed9f-480e-ba91-756407fa1b9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:25.722468 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5b2160-ed9f-480e-ba91-756407fa1b9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:25.725258 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5b2160-ed9f-480e-ba91-756407fa1b9b service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:25.745940 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 865/1741] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:19:26.500678 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69d6aee-a487-4ad3-9d9f-a98da1b0977f service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:26.502162 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d69d6aee-a487-4ad3-9d9f-a98da1b0977f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:26.532004 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69d6aee-a487-4ad3-9d9f-a98da1b0977f service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:26.555740 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc99922-6506-4715-98bb-5eb4ffa070a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:26.556651 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc99922-6506-4715-98bb-5eb4ffa070a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:26.556936 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc99922-6506-4715-98bb-5eb4ffa070a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:26.557256 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc99922-6506-4715-98bb-5eb4ffa070a7 admin admin] PATH_INFO: `/v3/users/79baf062f47a459bb5cbc12de5df5f60` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:26.562007 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc99922-6506-4715-98bb-5eb4ffa070a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79baf062f47a459bb5cbc12de5df5f60)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:26.566624 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc99922-6506-4715-98bb-5eb4ffa070a7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:26.674837 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cfc99922-6506-4715-98bb-5eb4ffa070a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79baf062f47a459bb5cbc12de5df5f60'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:26.675900 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cfc99922-6506-4715-98bb-5eb4ffa070a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79baf062f47a459bb5cbc12de5df5f60'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:26.698649 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cfc99922-6506-4715-98bb-5eb4ffa070a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79baf062f47a459bb5cbc12de5df5f60'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:26.720487 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cfc99922-6506-4715-98bb-5eb4ffa070a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79baf062f47a459bb5cbc12de5df5f60'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:26.721895 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cfc99922-6506-4715-98bb-5eb4ffa070a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79baf062f47a459bb5cbc12de5df5f60'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:26.755474 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 877/1742] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:19:26 2022] DELETE /identity/v3/users/79baf062f47a459bb5cbc12de5df5f60 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:26.761353 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8accbc6-99b2-446e-9c23-dff752725332 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:26.762211 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8accbc6-99b2-446e-9c23-dff752725332 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:26.763105 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8accbc6-99b2-446e-9c23-dff752725332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:26.784977 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-d8accbc6-99b2-446e-9c23-dff752725332 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:26.786710 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d8accbc6-99b2-446e-9c23-dff752725332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:26.833585 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc99922-6506-4715-98bb-5eb4ffa070a7 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:26.836172 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cfc99922-6506-4715-98bb-5eb4ffa070a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:26.983822 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 866/1743] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:26.990618 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8accbc6-99b2-446e-9c23-dff752725332 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:26.993498 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d8accbc6-99b2-446e-9c23-dff752725332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:27.004182 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc99922-6506-4715-98bb-5eb4ffa070a7 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:27.015087 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feb1134-ba1e-4c74-8282-63ba0b204edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:27.017423 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb1134-ba1e-4c74-8282-63ba0b204edc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:27.018314 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb1134-ba1e-4c74-8282-63ba0b204edc admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:27.020076 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb1134-ba1e-4c74-8282-63ba0b204edc admin admin] PATH_INFO: `/v3/projects/7e55a370a92648a3a581bba404179ccc` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:27.022376 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feb1134-ba1e-4c74-8282-63ba0b204edc admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e55a370a92648a3a581bba404179ccc)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:27.025196 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feb1134-ba1e-4c74-8282-63ba0b204edc admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:27.030182 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:27.114914 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8accbc6-99b2-446e-9c23-dff752725332 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:27.135558 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7feb1134-ba1e-4c74-8282-63ba0b204edc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e55a370a92648a3a581bba404179ccc', 'request_id': 'req-7feb1134-ba1e-4c74-8282-63ba0b204edc'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:27.148812 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7feb1134-ba1e-4c74-8282-63ba0b204edc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e55a370a92648a3a581bba404179ccc', 'request_id': 'req-7feb1134-ba1e-4c74-8282-63ba0b204edc'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:27.159102 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7feb1134-ba1e-4c74-8282-63ba0b204edc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e55a370a92648a3a581bba404179ccc', 'request_id': 'req-7feb1134-ba1e-4c74-8282-63ba0b204edc'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:27.196170 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7feb1134-ba1e-4c74-8282-63ba0b204edc admin admin] The token cache is being invalidate because project 7e55a370a92648a3a581bba404179ccc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:19:27.196489 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7feb1134-ba1e-4c74-8282-63ba0b204edc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:27.198943 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7feb1134-ba1e-4c74-8282-63ba0b204edc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:27.204766 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff39e20-7041-4355-a40a-5be73dd99d84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:27.204931 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 878/1744] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:19:26 2022] DELETE /identity/v3/projects/7e55a370a92648a3a581bba404179ccc => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:27.208706 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feb1134-ba1e-4c74-8282-63ba0b204edc admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:27.209984 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff39e20-7041-4355-a40a-5be73dd99d84 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:27.210438 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff39e20-7041-4355-a40a-5be73dd99d84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:27.211023 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff39e20-7041-4355-a40a-5be73dd99d84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:27.213402 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-cff39e20-7041-4355-a40a-5be73dd99d84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:27.213525 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7feb1134-ba1e-4c74-8282-63ba0b204edc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:27.268835 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:27.336537 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feb1134-ba1e-4c74-8282-63ba0b204edc admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:27.336744 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff39e20-7041-4355-a40a-5be73dd99d84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:27.339119 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff39e20-7041-4355-a40a-5be73dd99d84 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:27.343376 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f91936-4825-4229-93a5-00b157aacee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:27.344483 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f91936-4825-4229-93a5-00b157aacee9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:27.344924 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f91936-4825-4229-93a5-00b157aacee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:27.345383 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f91936-4825-4229-93a5-00b157aacee9 admin admin] PATH_INFO: `/v3/users/98fff7c679bd49938f97d4adde536f52` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:27.347162 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f91936-4825-4229-93a5-00b157aacee9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98fff7c679bd49938f97d4adde536f52)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:27.349481 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f91936-4825-4229-93a5-00b157aacee9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:27.365192 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 867/1745] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:19:27.434596 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-35f91936-4825-4229-93a5-00b157aacee9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98fff7c679bd49938f97d4adde536f52'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:27.435469 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-35f91936-4825-4229-93a5-00b157aacee9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98fff7c679bd49938f97d4adde536f52'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:27.448742 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-35f91936-4825-4229-93a5-00b157aacee9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98fff7c679bd49938f97d4adde536f52'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:27.490608 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-35f91936-4825-4229-93a5-00b157aacee9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98fff7c679bd49938f97d4adde536f52'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:27.494961 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-35f91936-4825-4229-93a5-00b157aacee9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98fff7c679bd49938f97d4adde536f52'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:27.522172 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 879/1746] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:19:27 2022] DELETE /identity/v3/users/98fff7c679bd49938f97d4adde536f52 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:19:27.580548 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff39e20-7041-4355-a40a-5be73dd99d84 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:27.582720 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-cff39e20-7041-4355-a40a-5be73dd99d84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:27.600203 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff39e20-7041-4355-a40a-5be73dd99d84 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:27.685203 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035af3e1-cad3-43fd-b6d6-ed1760168e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:27.686317 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035af3e1-cad3-43fd-b6d6-ed1760168e30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:27.686609 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035af3e1-cad3-43fd-b6d6-ed1760168e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:27.686907 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035af3e1-cad3-43fd-b6d6-ed1760168e30 admin admin] PATH_INFO: `/v3/projects/35e6378c14874946aa7131a65a714d5e` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:27.688254 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035af3e1-cad3-43fd-b6d6-ed1760168e30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35e6378c14874946aa7131a65a714d5e)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:27.692059 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035af3e1-cad3-43fd-b6d6-ed1760168e30 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:27.759051 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-035af3e1-cad3-43fd-b6d6-ed1760168e30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35e6378c14874946aa7131a65a714d5e', 'request_id': 'req-035af3e1-cad3-43fd-b6d6-ed1760168e30'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:27.767733 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-035af3e1-cad3-43fd-b6d6-ed1760168e30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35e6378c14874946aa7131a65a714d5e', 'request_id': 'req-035af3e1-cad3-43fd-b6d6-ed1760168e30'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:27.775837 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-035af3e1-cad3-43fd-b6d6-ed1760168e30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35e6378c14874946aa7131a65a714d5e', 'request_id': 'req-035af3e1-cad3-43fd-b6d6-ed1760168e30'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:27.798385 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-035af3e1-cad3-43fd-b6d6-ed1760168e30 admin admin] The token cache is being invalidate because project 35e6378c14874946aa7131a65a714d5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:19:27.798897 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-035af3e1-cad3-43fd-b6d6-ed1760168e30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:27.799728 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-035af3e1-cad3-43fd-b6d6-ed1760168e30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:27.802136 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 868/1747] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:19:27 2022] DELETE /identity/v3/projects/35e6378c14874946aa7131a65a714d5e => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:27.806483 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f91936-4825-4229-93a5-00b157aacee9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:27.810278 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-35f91936-4825-4229-93a5-00b157aacee9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:27.827017 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f91936-4825-4229-93a5-00b157aacee9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:27.833710 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:27.935861 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a511f156-1e76-4ea0-85cc-4c1bbf7726c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:27.937550 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a511f156-1e76-4ea0-85cc-4c1bbf7726c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:27.945923 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a511f156-1e76-4ea0-85cc-4c1bbf7726c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:27.946616 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a511f156-1e76-4ea0-85cc-4c1bbf7726c3 admin admin] PATH_INFO: `/v3/users/df4a0d9f18044b4485b2e85284a1d4d3` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:27.948267 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a511f156-1e76-4ea0-85cc-4c1bbf7726c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df4a0d9f18044b4485b2e85284a1d4d3)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:27.967017 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a511f156-1e76-4ea0-85cc-4c1bbf7726c3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:28.183501 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a511f156-1e76-4ea0-85cc-4c1bbf7726c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df4a0d9f18044b4485b2e85284a1d4d3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:28.195098 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a511f156-1e76-4ea0-85cc-4c1bbf7726c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df4a0d9f18044b4485b2e85284a1d4d3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:28.257003 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a511f156-1e76-4ea0-85cc-4c1bbf7726c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df4a0d9f18044b4485b2e85284a1d4d3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:28.296653 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a511f156-1e76-4ea0-85cc-4c1bbf7726c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df4a0d9f18044b4485b2e85284a1d4d3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:28.299823 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a511f156-1e76-4ea0-85cc-4c1bbf7726c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df4a0d9f18044b4485b2e85284a1d4d3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:28.361936 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 880/1748] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:19:27 2022] DELETE /identity/v3/users/df4a0d9f18044b4485b2e85284a1d4d3 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:28.736446 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01da66-9fd8-45dc-8071-f131cacefd7b None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:28.736735 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01da66-9fd8-45dc-8071-f131cacefd7b None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:28.737069 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01da66-9fd8-45dc-8071-f131cacefd7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:28.781774 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-5a01da66-9fd8-45dc-8071-f131cacefd7b None None] MFA Rules not processed for user `78668b939c5847b1b73e9c28b5d8db7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:28.784490 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-5a01da66-9fd8-45dc-8071-f131cacefd7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:28.931469 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 869/1749] 10.222.0.71 () {58 vars in 1009 bytes} [Mon Jul 4 09:19:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:29.021248 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff499fad-418a-4659-bac4-882d932fdbf1 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:29.021812 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff499fad-418a-4659-bac4-882d932fdbf1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:29.022126 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff499fad-418a-4659-bac4-882d932fdbf1 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:29.024599 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 881/1750] 10.222.0.71 () {54 vars in 921 bytes} [Mon Jul 4 09:19:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:19:29.064472 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd79cad5-a6fe-4c04-9d1c-24d2d2a20c54 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:29.064890 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd79cad5-a6fe-4c04-9d1c-24d2d2a20c54 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:29.065335 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd79cad5-a6fe-4c04-9d1c-24d2d2a20c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:29.078567 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:29.112981 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-cd79cad5-a6fe-4c04-9d1c-24d2d2a20c54 None None] MFA Rules not processed for user `293c695f15c8437d81a619f840318f44`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:29.116160 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-cd79cad5-a6fe-4c04-9d1c-24d2d2a20c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:29.202854 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff499fad-418a-4659-bac4-882d932fdbf1 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:29.204823 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ff499fad-418a-4659-bac4-882d932fdbf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:29.272598 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 870/1751] 10.222.0.71 () {58 vars in 1018 bytes} [Mon Jul 4 09:19:29 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:29.405293 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff499fad-418a-4659-bac4-882d932fdbf1 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:29.411069 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceef23b-9eb5-4fb3-be35-2221a45ae368 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88b41b55769a4c449716c3590ed58225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:29.411747 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceef23b-9eb5-4fb3-be35-2221a45ae368 service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:29.412181 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceef23b-9eb5-4fb3-be35-2221a45ae368 service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:29.412935 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceef23b-9eb5-4fb3-be35-2221a45ae368 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:29.427700 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceef23b-9eb5-4fb3-be35-2221a45ae368 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:29.429551 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceef23b-9eb5-4fb3-be35-2221a45ae368 service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:29.446941 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 882/1752] 10.222.0.71 () {60 vars in 1384 bytes} [Mon Jul 4 09:19:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 04 09:19:29.762674 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd79cad5-a6fe-4c04-9d1c-24d2d2a20c54 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:29.764123 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-cd79cad5-a6fe-4c04-9d1c-24d2d2a20c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:29.808521 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd79cad5-a6fe-4c04-9d1c-24d2d2a20c54 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:29.844968 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce82713-ff4e-4c00-8555-5869a1edefda service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '293c695f15c8437d81a619f840318f44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:29.851349 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce82713-ff4e-4c00-8555-5869a1edefda service placement] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:29.851853 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce82713-ff4e-4c00-8555-5869a1edefda service placement] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:29.852356 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce82713-ff4e-4c00-8555-5869a1edefda service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:29.888873 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce82713-ff4e-4c00-8555-5869a1edefda service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:29.890758 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce82713-ff4e-4c00-8555-5869a1edefda service placement] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:29.928615 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 871/1753] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:19:30.140345 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceef23b-9eb5-4fb3-be35-2221a45ae368 service swift] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:30.142658 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2ceef23b-9eb5-4fb3-be35-2221a45ae368 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:30.214797 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:30.371630 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab3ea8-7470-4015-a70f-73afc6160218 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:30.372514 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab3ea8-7470-4015-a70f-73afc6160218 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:30.373387 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab3ea8-7470-4015-a70f-73afc6160218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:30.373594 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceef23b-9eb5-4fb3-be35-2221a45ae368 service swift] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:30.403954 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-c1ab3ea8-7470-4015-a70f-73afc6160218 None None] MFA Rules not processed for user `e50f9b53a10a43fa9b5fec03d01f451f`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:30.407791 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c1ab3ea8-7470-4015-a70f-73afc6160218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:30.412660 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9096af2-cb0f-4c11-a88e-249e8ae2f213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:30.414286 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9096af2-cb0f-4c11-a88e-249e8ae2f213 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:30.414730 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9096af2-cb0f-4c11-a88e-249e8ae2f213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:30.415055 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9096af2-cb0f-4c11-a88e-249e8ae2f213 admin admin] PATH_INFO: `/v3/users/3ec0ef5374ad40e0831ac494b5ea5b11` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:30.419542 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9096af2-cb0f-4c11-a88e-249e8ae2f213 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ec0ef5374ad40e0831ac494b5ea5b11)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:30.428341 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9096af2-cb0f-4c11-a88e-249e8ae2f213 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:30.531474 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 872/1754] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:30.531474 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ab3ea8-7470-4015-a70f-73afc6160218 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:30.531766 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e9096af2-cb0f-4c11-a88e-249e8ae2f213 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ec0ef5374ad40e0831ac494b5ea5b11'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:30.532504 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e9096af2-cb0f-4c11-a88e-249e8ae2f213 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ec0ef5374ad40e0831ac494b5ea5b11'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:30.542817 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e9096af2-cb0f-4c11-a88e-249e8ae2f213 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ec0ef5374ad40e0831ac494b5ea5b11'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:30.543980 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c1ab3ea8-7470-4015-a70f-73afc6160218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:30.563811 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e9096af2-cb0f-4c11-a88e-249e8ae2f213 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ec0ef5374ad40e0831ac494b5ea5b11'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:30.564956 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e9096af2-cb0f-4c11-a88e-249e8ae2f213 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ec0ef5374ad40e0831ac494b5ea5b11'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:30.587557 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 883/1755] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:19:30 2022] DELETE /identity/v3/users/3ec0ef5374ad40e0831ac494b5ea5b11 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:30.588532 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9096af2-cb0f-4c11-a88e-249e8ae2f213 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:30.590465 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e9096af2-cb0f-4c11-a88e-249e8ae2f213 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:30.591788 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ab3ea8-7470-4015-a70f-73afc6160218 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:30.731895 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9096af2-cb0f-4c11-a88e-249e8ae2f213 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:30.746979 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f673ab-9b1e-4a3c-ad43-a2a070f37e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:30.748383 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f673ab-9b1e-4a3c-ad43-a2a070f37e39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:30.748680 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f673ab-9b1e-4a3c-ad43-a2a070f37e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:30.748961 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f673ab-9b1e-4a3c-ad43-a2a070f37e39 admin admin] PATH_INFO: `/v3/projects/d718667220734fec99f25f3850dfcbf5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:30.754035 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f673ab-9b1e-4a3c-ad43-a2a070f37e39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d718667220734fec99f25f3850dfcbf5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:30.756047 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f673ab-9b1e-4a3c-ad43-a2a070f37e39 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:30.794646 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56af4487-73ca-4bae-b218-f1bad25a8eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:30.795202 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af4487-73ca-4bae-b218-f1bad25a8eb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:30.795574 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af4487-73ca-4bae-b218-f1bad25a8eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:30.795983 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af4487-73ca-4bae-b218-f1bad25a8eb6 admin admin] PATH_INFO: `/v3/projects/b18081aaec9c4dbd97844751037c7aa3` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:30.798163 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56af4487-73ca-4bae-b218-f1bad25a8eb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b18081aaec9c4dbd97844751037c7aa3)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:30.803331 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56af4487-73ca-4bae-b218-f1bad25a8eb6 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:30.893909 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:30.910230 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-63f673ab-9b1e-4a3c-ad43-a2a070f37e39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd718667220734fec99f25f3850dfcbf5', 'request_id': 'req-63f673ab-9b1e-4a3c-ad43-a2a070f37e39'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:30.923452 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-63f673ab-9b1e-4a3c-ad43-a2a070f37e39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd718667220734fec99f25f3850dfcbf5', 'request_id': 'req-63f673ab-9b1e-4a3c-ad43-a2a070f37e39'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:30.947358 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:30.947910 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-63f673ab-9b1e-4a3c-ad43-a2a070f37e39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd718667220734fec99f25f3850dfcbf5', 'request_id': 'req-63f673ab-9b1e-4a3c-ad43-a2a070f37e39'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:30.996415 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-56af4487-73ca-4bae-b218-f1bad25a8eb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b18081aaec9c4dbd97844751037c7aa3', 'request_id': 'req-56af4487-73ca-4bae-b218-f1bad25a8eb6'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:30.999800 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-63f673ab-9b1e-4a3c-ad43-a2a070f37e39 admin admin] The token cache is being invalidate because project d718667220734fec99f25f3850dfcbf5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:19:31.000154 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-63f673ab-9b1e-4a3c-ad43-a2a070f37e39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:31.002764 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-63f673ab-9b1e-4a3c-ad43-a2a070f37e39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:31.014433 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 884/1756] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:19:30 2022] DELETE /identity/v3/projects/d718667220734fec99f25f3850dfcbf5 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:31.020338 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f673ab-9b1e-4a3c-ad43-a2a070f37e39 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:31.034279 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-63f673ab-9b1e-4a3c-ad43-a2a070f37e39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:31.046409 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-56af4487-73ca-4bae-b218-f1bad25a8eb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b18081aaec9c4dbd97844751037c7aa3', 'request_id': 'req-56af4487-73ca-4bae-b218-f1bad25a8eb6'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:31.083450 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-56af4487-73ca-4bae-b218-f1bad25a8eb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b18081aaec9c4dbd97844751037c7aa3', 'request_id': 'req-56af4487-73ca-4bae-b218-f1bad25a8eb6'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:31.119260 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-56af4487-73ca-4bae-b218-f1bad25a8eb6 admin admin] The token cache is being invalidate because project b18081aaec9c4dbd97844751037c7aa3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:19:31.120116 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-56af4487-73ca-4bae-b218-f1bad25a8eb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:31.121454 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-56af4487-73ca-4bae-b218-f1bad25a8eb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:31.126591 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 873/1757] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:19:30 2022] DELETE /identity/v3/projects/b18081aaec9c4dbd97844751037c7aa3 => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:31.128816 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56af4487-73ca-4bae-b218-f1bad25a8eb6 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:31.134630 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-56af4487-73ca-4bae-b218-f1bad25a8eb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:31.200945 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f673ab-9b1e-4a3c-ad43-a2a070f37e39 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:31.342741 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd99b569-b095-4e78-8157-a3282ee0ef82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:31.344085 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd99b569-b095-4e78-8157-a3282ee0ef82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:31.344615 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd99b569-b095-4e78-8157-a3282ee0ef82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:31.345852 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd99b569-b095-4e78-8157-a3282ee0ef82 admin admin] PATH_INFO: `/v3/projects/8b1abe013be34290b713c67f84aa0d1c` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:31.356078 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd99b569-b095-4e78-8157-a3282ee0ef82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b1abe013be34290b713c67f84aa0d1c)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:31.360135 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56af4487-73ca-4bae-b218-f1bad25a8eb6 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:31.363461 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd99b569-b095-4e78-8157-a3282ee0ef82 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:31.365499 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed6b71d-15fa-4e52-ad13-9308ae750d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:31.366491 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6b71d-15fa-4e52-ad13-9308ae750d1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:31.367388 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6b71d-15fa-4e52-ad13-9308ae750d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:31.367733 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6b71d-15fa-4e52-ad13-9308ae750d1f admin admin] PATH_INFO: `/v3/users/e3035d47ea4f4fe8af7107fc5ffa63bb` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:31.369736 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed6b71d-15fa-4e52-ad13-9308ae750d1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3035d47ea4f4fe8af7107fc5ffa63bb)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:31.371971 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed6b71d-15fa-4e52-ad13-9308ae750d1f admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:31.523436 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-1ed6b71d-15fa-4e52-ad13-9308ae750d1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3035d47ea4f4fe8af7107fc5ffa63bb'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:31.546927 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-1ed6b71d-15fa-4e52-ad13-9308ae750d1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3035d47ea4f4fe8af7107fc5ffa63bb'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:31.547916 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-1ed6b71d-15fa-4e52-ad13-9308ae750d1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3035d47ea4f4fe8af7107fc5ffa63bb'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:31.584782 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-1ed6b71d-15fa-4e52-ad13-9308ae750d1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3035d47ea4f4fe8af7107fc5ffa63bb'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:31.621522 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-1ed6b71d-15fa-4e52-ad13-9308ae750d1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3035d47ea4f4fe8af7107fc5ffa63bb'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:31.635450 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 874/1758] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:19:31 2022] DELETE /identity/v3/users/e3035d47ea4f4fe8af7107fc5ffa63bb => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:31.637707 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df49a765-0f47-47a1-a635-a070a4b1244a None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:31.640354 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df49a765-0f47-47a1-a635-a070a4b1244a None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:31.640665 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df49a765-0f47-47a1-a635-a070a4b1244a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:31.691670 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-df49a765-0f47-47a1-a635-a070a4b1244a None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:31.693183 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-df49a765-0f47-47a1-a635-a070a4b1244a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:31.783340 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:31.823887 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dd99b569-b095-4e78-8157-a3282ee0ef82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b1abe013be34290b713c67f84aa0d1c', 'request_id': 'req-dd99b569-b095-4e78-8157-a3282ee0ef82'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:31.826682 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 875/1759] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:31.834703 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd972c2-ead3-490e-9748-987153e7a110 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:31.837163 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd972c2-ead3-490e-9748-987153e7a110 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:31.837743 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd972c2-ead3-490e-9748-987153e7a110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:31.846098 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dd99b569-b095-4e78-8157-a3282ee0ef82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b1abe013be34290b713c67f84aa0d1c', 'request_id': 'req-dd99b569-b095-4e78-8157-a3282ee0ef82'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:31.888122 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dd99b569-b095-4e78-8157-a3282ee0ef82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b1abe013be34290b713c67f84aa0d1c', 'request_id': 'req-dd99b569-b095-4e78-8157-a3282ee0ef82'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:31.908668 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-0cd972c2-ead3-490e-9748-987153e7a110 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:31.910678 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0cd972c2-ead3-490e-9748-987153e7a110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:31.923563 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dd99b569-b095-4e78-8157-a3282ee0ef82 admin admin] The token cache is being invalidate because project 8b1abe013be34290b713c67f84aa0d1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:19:31.924126 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dd99b569-b095-4e78-8157-a3282ee0ef82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:31.925177 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dd99b569-b095-4e78-8157-a3282ee0ef82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:31.927618 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 885/1760] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:19:31 2022] DELETE /identity/v3/projects/8b1abe013be34290b713c67f84aa0d1c => generated 0 bytes in 913 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:31.933524 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378dd3ff-34f5-4d0d-aebf-c25ca1945b5e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:31.934459 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378dd3ff-34f5-4d0d-aebf-c25ca1945b5e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:31.935611 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378dd3ff-34f5-4d0d-aebf-c25ca1945b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:31.970241 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:32.032594 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-378dd3ff-34f5-4d0d-aebf-c25ca1945b5e None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:32.034175 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-378dd3ff-34f5-4d0d-aebf-c25ca1945b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:32.068240 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 876/1761] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:32.069162 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd972c2-ead3-490e-9748-987153e7a110 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:32.072121 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0cd972c2-ead3-490e-9748-987153e7a110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:32.101274 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd972c2-ead3-490e-9748-987153e7a110 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:32.116370 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e485e742-2fb2-4640-8ec0-02fd4c01a253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:32.122653 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e485e742-2fb2-4640-8ec0-02fd4c01a253 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:32.123647 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e485e742-2fb2-4640-8ec0-02fd4c01a253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:32.124413 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e485e742-2fb2-4640-8ec0-02fd4c01a253 admin admin] PATH_INFO: `/v3/projects/e3bf7211de5f483e8a55f4ca38246f2c` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:32.127425 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e485e742-2fb2-4640-8ec0-02fd4c01a253 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3bf7211de5f483e8a55f4ca38246f2c)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:32.133334 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e485e742-2fb2-4640-8ec0-02fd4c01a253 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:32.207708 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 886/1762] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:32.208888 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378dd3ff-34f5-4d0d-aebf-c25ca1945b5e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:32.246495 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e485e742-2fb2-4640-8ec0-02fd4c01a253 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3bf7211de5f483e8a55f4ca38246f2c', 'request_id': 'req-e485e742-2fb2-4640-8ec0-02fd4c01a253'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:32.250073 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378dd3ff-34f5-4d0d-aebf-c25ca1945b5e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:32.262666 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e485e742-2fb2-4640-8ec0-02fd4c01a253 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3bf7211de5f483e8a55f4ca38246f2c', 'request_id': 'req-e485e742-2fb2-4640-8ec0-02fd4c01a253'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:32.281310 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e485e742-2fb2-4640-8ec0-02fd4c01a253 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3bf7211de5f483e8a55f4ca38246f2c', 'request_id': 'req-e485e742-2fb2-4640-8ec0-02fd4c01a253'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:32.309251 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e485e742-2fb2-4640-8ec0-02fd4c01a253 admin admin] The token cache is being invalidate because project e3bf7211de5f483e8a55f4ca38246f2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:19:32.309558 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e485e742-2fb2-4640-8ec0-02fd4c01a253 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:32.313491 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e485e742-2fb2-4640-8ec0-02fd4c01a253 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:32.319782 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 877/1763] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:19:32 2022] DELETE /identity/v3/projects/e3bf7211de5f483e8a55f4ca38246f2c => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:32.321872 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327b8626-2252-4afd-9db6-00772444ed1e None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:32.322168 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327b8626-2252-4afd-9db6-00772444ed1e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:32.322459 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327b8626-2252-4afd-9db6-00772444ed1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:32.404432 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-327b8626-2252-4afd-9db6-00772444ed1e None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:32.412517 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-327b8626-2252-4afd-9db6-00772444ed1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:32.521778 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28986e0-907b-49b4-ad81-04c637fb3a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:32.522628 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28986e0-907b-49b4-ad81-04c637fb3a9f admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:32.523140 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28986e0-907b-49b4-ad81-04c637fb3a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:32.524020 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28986e0-907b-49b4-ad81-04c637fb3a9f admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:32.525474 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28986e0-907b-49b4-ad81-04c637fb3a9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:19:32.525862 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28986e0-907b-49b4-ad81-04c637fb3a9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:32.528471 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28986e0-907b-49b4-ad81-04c637fb3a9f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:32.569279 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 887/1764] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:19:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:19:32.570808 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28986e0-907b-49b4-ad81-04c637fb3a9f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:32.589974 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 878/1765] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:32.594932 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327b8626-2252-4afd-9db6-00772444ed1e None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:32.597957 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-327b8626-2252-4afd-9db6-00772444ed1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:32.601223 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e28986e0-907b-49b4-ad81-04c637fb3a9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:32.630305 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:32.639612 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327b8626-2252-4afd-9db6-00772444ed1e None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:32.651517 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28986e0-907b-49b4-ad81-04c637fb3a9f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:32.658754 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbcba80-2a07-4b93-8a11-f10fd6381014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:32.659456 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbcba80-2a07-4b93-8a11-f10fd6381014 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:32.659747 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbcba80-2a07-4b93-8a11-f10fd6381014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:32.660034 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbcba80-2a07-4b93-8a11-f10fd6381014 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:32.660779 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbcba80-2a07-4b93-8a11-f10fd6381014 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:32.662770 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbcba80-2a07-4b93-8a11-f10fd6381014 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:32.668220 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81137e3b-46ae-46b7-8906-ae21d39b69e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:32.675811 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81137e3b-46ae-46b7-8906-ae21d39b69e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:32.676452 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81137e3b-46ae-46b7-8906-ae21d39b69e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:32.677561 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81137e3b-46ae-46b7-8906-ae21d39b69e6 admin admin] PATH_INFO: `/v3/users/ed486d69757b4f50996af383aeecd90b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:32.681446 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81137e3b-46ae-46b7-8906-ae21d39b69e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed486d69757b4f50996af383aeecd90b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:32.692867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81137e3b-46ae-46b7-8906-ae21d39b69e6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:32.747964 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 879/1766] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:19:32 2022] POST /identity/v3/projects => generated 401 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:19:32.748921 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbcba80-2a07-4b93-8a11-f10fd6381014 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:32.790080 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-81137e3b-46ae-46b7-8906-ae21d39b69e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed486d69757b4f50996af383aeecd90b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:32.791535 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-81137e3b-46ae-46b7-8906-ae21d39b69e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed486d69757b4f50996af383aeecd90b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:32.820978 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-81137e3b-46ae-46b7-8906-ae21d39b69e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed486d69757b4f50996af383aeecd90b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:32.848670 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-81137e3b-46ae-46b7-8906-ae21d39b69e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed486d69757b4f50996af383aeecd90b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:32.850236 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-81137e3b-46ae-46b7-8906-ae21d39b69e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed486d69757b4f50996af383aeecd90b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:32.876131 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbcba80-2a07-4b93-8a11-f10fd6381014 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:32.878404 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 888/1767] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:19:32 2022] DELETE /identity/v3/users/ed486d69757b4f50996af383aeecd90b => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:32.880255 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81137e3b-46ae-46b7-8906-ae21d39b69e6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:32.883146 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-81137e3b-46ae-46b7-8906-ae21d39b69e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:32.914023 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:32.993376 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102477e4-a67d-40c3-9439-e878beadd554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:32.994064 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102477e4-a67d-40c3-9439-e878beadd554 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:32.994565 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102477e4-a67d-40c3-9439-e878beadd554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:32.994856 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102477e4-a67d-40c3-9439-e878beadd554 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:32.995954 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102477e4-a67d-40c3-9439-e878beadd554 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:19:32.996564 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102477e4-a67d-40c3-9439-e878beadd554 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:32.999570 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102477e4-a67d-40c3-9439-e878beadd554 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:33.011284 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81137e3b-46ae-46b7-8906-ae21d39b69e6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:33.024831 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 880/1768] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:19:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:19:33.030008 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102477e4-a67d-40c3-9439-e878beadd554 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:33.032141 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-102477e4-a67d-40c3-9439-e878beadd554 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:33.033569 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb92c5e-624d-4b0f-9c30-15908c06fcb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:33.034703 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb92c5e-624d-4b0f-9c30-15908c06fcb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:33.035194 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb92c5e-624d-4b0f-9c30-15908c06fcb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:33.036314 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb92c5e-624d-4b0f-9c30-15908c06fcb6 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:33.037336 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb92c5e-624d-4b0f-9c30-15908c06fcb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:33.040447 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb92c5e-624d-4b0f-9c30-15908c06fcb6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:33.047889 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102477e4-a67d-40c3-9439-e878beadd554 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:33.054361 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2535f7fa-c2f4-40c5-b336-add125f03305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:33.055319 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2535f7fa-c2f4-40c5-b336-add125f03305 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:33.055613 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2535f7fa-c2f4-40c5-b336-add125f03305 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:33.056224 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2535f7fa-c2f4-40c5-b336-add125f03305 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:33.057496 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2535f7fa-c2f4-40c5-b336-add125f03305 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:33.060512 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2535f7fa-c2f4-40c5-b336-add125f03305 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:33.075367 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 889/1769] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:19:32 2022] POST /identity/v3/users => generated 301 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:19:33.076847 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb92c5e-624d-4b0f-9c30-15908c06fcb6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:33.092729 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb92c5e-624d-4b0f-9c30-15908c06fcb6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:33.099621 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa234c7-ff38-4dc3-b38d-07f08c0ff9f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:33.100536 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa234c7-ff38-4dc3-b38d-07f08c0ff9f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:33.100948 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa234c7-ff38-4dc3-b38d-07f08c0ff9f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:33.101618 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa234c7-ff38-4dc3-b38d-07f08c0ff9f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:33.102507 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa234c7-ff38-4dc3-b38d-07f08c0ff9f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:33.104952 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa234c7-ff38-4dc3-b38d-07f08c0ff9f0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:33.122561 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 890/1770] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:33 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:19:33.123640 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa234c7-ff38-4dc3-b38d-07f08c0ff9f0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:33.131150 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 881/1771] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:19:33 2022] POST /identity/v3/projects => generated 363 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:19:33.133968 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2535f7fa-c2f4-40c5-b336-add125f03305 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:33.139828 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa234c7-ff38-4dc3-b38d-07f08c0ff9f0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:33.268968 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc567386-eb01-4f83-8d38-a453179530ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:33.270430 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc567386-eb01-4f83-8d38-a453179530ae admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:33.271638 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc567386-eb01-4f83-8d38-a453179530ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:33.272144 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc567386-eb01-4f83-8d38-a453179530ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:33.275009 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc567386-eb01-4f83-8d38-a453179530ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:33.277522 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc567386-eb01-4f83-8d38-a453179530ae admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:33.315530 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 891/1772] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:33 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:19:33.317840 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc567386-eb01-4f83-8d38-a453179530ae admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:33.353852 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc567386-eb01-4f83-8d38-a453179530ae admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:33.369077 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa98c48-0c87-42f4-875c-445a10740fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:33.370460 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa98c48-0c87-42f4-875c-445a10740fab admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:33.372088 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa98c48-0c87-42f4-875c-445a10740fab admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:33.372411 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2535f7fa-c2f4-40c5-b336-add125f03305 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:33.372960 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa98c48-0c87-42f4-875c-445a10740fab admin admin] PATH_INFO: `/v3/projects/f96f099eae0c4fe58ef30986514abbb8/users/38842d0848054b2dbd14fe07dd0fef97/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:33.410291 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e713fc-4410-47b6-af1f-3c09320de868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:33.411875 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e713fc-4410-47b6-af1f-3c09320de868 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:33.412347 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e713fc-4410-47b6-af1f-3c09320de868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:33.412941 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e713fc-4410-47b6-af1f-3c09320de868 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:33.413446 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e713fc-4410-47b6-af1f-3c09320de868 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:33.417322 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa98c48-0c87-42f4-875c-445a10740fab admin admin] RBAC: Authorizing `identity:create_grant(project_id=f96f099eae0c4fe58ef30986514abbb8, user_id=38842d0848054b2dbd14fe07dd0fef97, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:33.420978 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e713fc-4410-47b6-af1f-3c09320de868 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:33.423527 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa98c48-0c87-42f4-875c-445a10740fab admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:33.431601 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:33.466116 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 892/1773] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:19:33 2022] PUT /identity/v3/projects/f96f099eae0c4fe58ef30986514abbb8/users/38842d0848054b2dbd14fe07dd0fef97/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:33.466479 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 882/1774] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:19:33 2022] POST /identity/v3/users => generated 282 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:19:33.494091 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da441090-3bae-49fb-8897-2f1f099ea517 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:33.494379 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da441090-3bae-49fb-8897-2f1f099ea517 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:33.494592 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa98c48-0c87-42f4-875c-445a10740fab admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:33.494788 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da441090-3bae-49fb-8897-2f1f099ea517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:33.568532 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-da441090-3bae-49fb-8897-2f1f099ea517 None None] MFA Rules not processed for user `38842d0848054b2dbd14fe07dd0fef97`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:33.569769 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-da441090-3bae-49fb-8897-2f1f099ea517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:33.691549 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa98c48-0c87-42f4-875c-445a10740fab admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:33.700605 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 883/1775] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:33 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:33.706749 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da441090-3bae-49fb-8897-2f1f099ea517 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:33.710363 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fbe88b-ee3b-464b-9583-236e0406eba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:33.711666 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fbe88b-ee3b-464b-9583-236e0406eba8 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:33.711951 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fbe88b-ee3b-464b-9583-236e0406eba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:33.712229 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fbe88b-ee3b-464b-9583-236e0406eba8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:33.713217 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fbe88b-ee3b-464b-9583-236e0406eba8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:33.715116 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fbe88b-ee3b-464b-9583-236e0406eba8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:33.717938 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:33.722603 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da441090-3bae-49fb-8897-2f1f099ea517 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:33.730324 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f0ed7d-530c-4e4a-84ed-057c04b4cd29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:33.731279 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f0ed7d-530c-4e4a-84ed-057c04b4cd29 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:33.731725 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f0ed7d-530c-4e4a-84ed-057c04b4cd29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:33.732162 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f0ed7d-530c-4e4a-84ed-057c04b4cd29 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:33.733084 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f0ed7d-530c-4e4a-84ed-057c04b4cd29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:33.735225 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f0ed7d-530c-4e4a-84ed-057c04b4cd29 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:33.739711 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 893/1776] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:33 2022] GET /identity/v3/roles => generated 1420 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:19:33.745462 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fbe88b-ee3b-464b-9583-236e0406eba8 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:33.757454 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fbe88b-ee3b-464b-9583-236e0406eba8 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:33.782872 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374eceac-5b28-4999-8736-198ef6cc9615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:33.787618 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 884/1777] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:19:33 2022] POST /identity/v3/projects => generated 395 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:19:33.789545 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f0ed7d-530c-4e4a-84ed-057c04b4cd29 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:33.790644 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374eceac-5b28-4999-8736-198ef6cc9615 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:33.790959 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374eceac-5b28-4999-8736-198ef6cc9615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:33.791248 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374eceac-5b28-4999-8736-198ef6cc9615 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:33.792849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374eceac-5b28-4999-8736-198ef6cc9615 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:33.794528 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374eceac-5b28-4999-8736-198ef6cc9615 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:33.833644 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 894/1778] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:33 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:19:33.834573 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374eceac-5b28-4999-8736-198ef6cc9615 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:33.863977 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374eceac-5b28-4999-8736-198ef6cc9615 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:33.903040 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f0ed7d-530c-4e4a-84ed-057c04b4cd29 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:33.917559 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cbf1ff-b3bd-4ef3-8fbe-3db5cb833b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:33.918432 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cbf1ff-b3bd-4ef3-8fbe-3db5cb833b59 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:33.918727 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cbf1ff-b3bd-4ef3-8fbe-3db5cb833b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:33.919015 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cbf1ff-b3bd-4ef3-8fbe-3db5cb833b59 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:33.919751 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cbf1ff-b3bd-4ef3-8fbe-3db5cb833b59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:33.926146 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cbf1ff-b3bd-4ef3-8fbe-3db5cb833b59 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:34.037831 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 885/1779] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:19:33 2022] POST /identity/v3/users => generated 304 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:19:34.062751 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cbf1ff-b3bd-4ef3-8fbe-3db5cb833b59 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:34.066750 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84f7a7c-b933-40af-8646-44b87a403fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:34.068580 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f7a7c-b933-40af-8646-44b87a403fcb admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:34.088035 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f7a7c-b933-40af-8646-44b87a403fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:34.088237 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f7a7c-b933-40af-8646-44b87a403fcb admin admin] PATH_INFO: `/v3/projects/8a46c06e19854d13a197aa13cc2a75f4/users/f3864d1503914ca0a6140cedabee06db/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:34.098647 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cbf1ff-b3bd-4ef3-8fbe-3db5cb833b59 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:34.116443 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc14efdd-bf08-431b-bf2c-8afcce4c7f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:34.118691 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14efdd-bf08-431b-bf2c-8afcce4c7f88 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:34.120089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14efdd-bf08-431b-bf2c-8afcce4c7f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:34.120957 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14efdd-bf08-431b-bf2c-8afcce4c7f88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:34.121953 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc14efdd-bf08-431b-bf2c-8afcce4c7f88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:34.129028 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc14efdd-bf08-431b-bf2c-8afcce4c7f88 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:34.134197 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84f7a7c-b933-40af-8646-44b87a403fcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a46c06e19854d13a197aa13cc2a75f4, user_id=f3864d1503914ca0a6140cedabee06db, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:34.136905 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84f7a7c-b933-40af-8646-44b87a403fcb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:34.182940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 895/1780] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:19:33 2022] PUT /identity/v3/projects/8a46c06e19854d13a197aa13cc2a75f4/users/f3864d1503914ca0a6140cedabee06db/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:34.194270 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 886/1781] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:34 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:19:34.196174 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84f7a7c-b933-40af-8646-44b87a403fcb admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:34.198105 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd291dd2-2606-4ebf-af36-641f18a900b5 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:34.198432 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd291dd2-2606-4ebf-af36-641f18a900b5 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:34.198704 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd291dd2-2606-4ebf-af36-641f18a900b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:34.267028 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-bd291dd2-2606-4ebf-af36-641f18a900b5 None None] MFA Rules not processed for user `f3864d1503914ca0a6140cedabee06db`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:34.271094 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-bd291dd2-2606-4ebf-af36-641f18a900b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:34.357817 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:34.482335 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84f7a7c-b933-40af-8646-44b87a403fcb admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:34.488880 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 887/1782] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:34 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:34.490563 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd291dd2-2606-4ebf-af36-641f18a900b5 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:34.508979 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc1a7f2-eae3-4341-a352-4885adb333bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:34.511351 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc1a7f2-eae3-4341-a352-4885adb333bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:34.511648 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc1a7f2-eae3-4341-a352-4885adb333bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:34.516514 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc1a7f2-eae3-4341-a352-4885adb333bb admin admin] PATH_INFO: `/v3/projects/1ac1170c339f4d5fa4a9ce7cbbdc21e4/users/7af5a088140e4809bb7cc9aa8060fd6d/roles/f36e053b442540f08041ac7878760f65` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:34.519601 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd291dd2-2606-4ebf-af36-641f18a900b5 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:34.537590 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0efd8c-8547-4cd9-a6e2-62889e2d7dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:34.538291 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0efd8c-8547-4cd9-a6e2-62889e2d7dd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:34.538644 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0efd8c-8547-4cd9-a6e2-62889e2d7dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:34.538934 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0efd8c-8547-4cd9-a6e2-62889e2d7dd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:34.540388 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0efd8c-8547-4cd9-a6e2-62889e2d7dd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:34.546141 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0efd8c-8547-4cd9-a6e2-62889e2d7dd6 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:34.594841 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc1a7f2-eae3-4341-a352-4885adb333bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ac1170c339f4d5fa4a9ce7cbbdc21e4, user_id=7af5a088140e4809bb7cc9aa8060fd6d, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:34.608496 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc1a7f2-eae3-4341-a352-4885adb333bb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:34.666974 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 896/1783] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:19:34 2022] PUT /identity/v3/projects/1ac1170c339f4d5fa4a9ce7cbbdc21e4/users/7af5a088140e4809bb7cc9aa8060fd6d/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:34.666974 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99748164-e277-40c3-aa05-c4a84a85095a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:34.666974 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99748164-e277-40c3-aa05-c4a84a85095a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:34.666974 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99748164-e277-40c3-aa05-c4a84a85095a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:34.717470 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 888/1784] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:19:34 2022] POST /identity/v3/projects => generated 365 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:19:34.718509 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0efd8c-8547-4cd9-a6e2-62889e2d7dd6 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:34.722326 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-99748164-e277-40c3-aa05-c4a84a85095a None None] MFA Rules not processed for user `15818e58661646f381da0973d4c2ff64`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:34.735546 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-99748164-e277-40c3-aa05-c4a84a85095a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:34.915984 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 897/1785] 10.222.0.65 () {58 vars in 1015 bytes} [Mon Jul 4 09:19:34 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:34.917394 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99748164-e277-40c3-aa05-c4a84a85095a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:34.919501 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-99748164-e277-40c3-aa05-c4a84a85095a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:34.943250 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0efd8c-8547-4cd9-a6e2-62889e2d7dd6 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:34.950905 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:34.970405 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99748164-e277-40c3-aa05-c4a84a85095a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:34.990331 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6581e107-c429-4866-a32a-926fdd02f78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:34.993522 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5d8668-df0a-4460-a9bf-91fa0b91e2c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:34.994619 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d8668-df0a-4460-a9bf-91fa0b91e2c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:34.994985 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d8668-df0a-4460-a9bf-91fa0b91e2c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:34.995267 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d8668-df0a-4460-a9bf-91fa0b91e2c9 admin admin] PATH_INFO: `/v3/projects/d93ad51f26d04cf3ba554fd45dc7116f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:34.996766 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5d8668-df0a-4460-a9bf-91fa0b91e2c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d93ad51f26d04cf3ba554fd45dc7116f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:35.001877 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5d8668-df0a-4460-a9bf-91fa0b91e2c9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:35.004517 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6581e107-c429-4866-a32a-926fdd02f78e admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:35.006368 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6581e107-c429-4866-a32a-926fdd02f78e admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:35.006849 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6581e107-c429-4866-a32a-926fdd02f78e admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:35.010121 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6581e107-c429-4866-a32a-926fdd02f78e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:35.016970 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6581e107-c429-4866-a32a-926fdd02f78e admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:35.057910 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 889/1786] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:34 2022] GET /identity/v3/roles => generated 1420 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:19:35.062033 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6581e107-c429-4866-a32a-926fdd02f78e admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:35.102448 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6581e107-c429-4866-a32a-926fdd02f78e admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:35.161019 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0f5d8668-df0a-4460-a9bf-91fa0b91e2c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd93ad51f26d04cf3ba554fd45dc7116f', 'request_id': 'req-0f5d8668-df0a-4460-a9bf-91fa0b91e2c9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:35.173969 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0f5d8668-df0a-4460-a9bf-91fa0b91e2c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd93ad51f26d04cf3ba554fd45dc7116f', 'request_id': 'req-0f5d8668-df0a-4460-a9bf-91fa0b91e2c9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:35.187571 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0f5d8668-df0a-4460-a9bf-91fa0b91e2c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd93ad51f26d04cf3ba554fd45dc7116f', 'request_id': 'req-0f5d8668-df0a-4460-a9bf-91fa0b91e2c9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:35.220157 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0f5d8668-df0a-4460-a9bf-91fa0b91e2c9 admin admin] The token cache is being invalidate because project d93ad51f26d04cf3ba554fd45dc7116f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:19:35.220379 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0f5d8668-df0a-4460-a9bf-91fa0b91e2c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:35.221304 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0f5d8668-df0a-4460-a9bf-91fa0b91e2c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:35.226020 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 898/1787] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:19:34 2022] DELETE /identity/v3/projects/d93ad51f26d04cf3ba554fd45dc7116f => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:35.230315 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5d8668-df0a-4460-a9bf-91fa0b91e2c9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:35.234508 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0f5d8668-df0a-4460-a9bf-91fa0b91e2c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:35.341650 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c7fddc-c575-4b82-8ad0-cca0f32f5296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:35.342856 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c7fddc-c575-4b82-8ad0-cca0f32f5296 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:35.343152 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c7fddc-c575-4b82-8ad0-cca0f32f5296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:35.343438 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c7fddc-c575-4b82-8ad0-cca0f32f5296 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:35.344153 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c7fddc-c575-4b82-8ad0-cca0f32f5296 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:35.350080 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c7fddc-c575-4b82-8ad0-cca0f32f5296 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:35.388480 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 890/1788] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:19:35 2022] POST /identity/v3/users => generated 289 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:19:35.389518 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c7fddc-c575-4b82-8ad0-cca0f32f5296 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:35.395572 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f9c7fddc-c575-4b82-8ad0-cca0f32f5296 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:35.401675 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5d8668-df0a-4460-a9bf-91fa0b91e2c9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:35.432530 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00210e09-4558-4c4b-9d42-e33b2ba92b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:35.433435 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00210e09-4558-4c4b-9d42-e33b2ba92b5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:35.433722 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00210e09-4558-4c4b-9d42-e33b2ba92b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:35.434005 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00210e09-4558-4c4b-9d42-e33b2ba92b5b admin admin] PATH_INFO: `/v3/projects/8d3a17536478469e80e0e431e9fa8b17` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:35.440283 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00210e09-4558-4c4b-9d42-e33b2ba92b5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d3a17536478469e80e0e431e9fa8b17)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:35.443081 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00210e09-4558-4c4b-9d42-e33b2ba92b5b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:35.520971 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:35.561495 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-00210e09-4558-4c4b-9d42-e33b2ba92b5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d3a17536478469e80e0e431e9fa8b17', 'request_id': 'req-00210e09-4558-4c4b-9d42-e33b2ba92b5b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:35.571909 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-00210e09-4558-4c4b-9d42-e33b2ba92b5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d3a17536478469e80e0e431e9fa8b17', 'request_id': 'req-00210e09-4558-4c4b-9d42-e33b2ba92b5b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:35.584495 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-00210e09-4558-4c4b-9d42-e33b2ba92b5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d3a17536478469e80e0e431e9fa8b17', 'request_id': 'req-00210e09-4558-4c4b-9d42-e33b2ba92b5b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:35.586950 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c7fddc-c575-4b82-8ad0-cca0f32f5296 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:35.604176 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-00210e09-4558-4c4b-9d42-e33b2ba92b5b admin admin] The token cache is being invalidate because project 8d3a17536478469e80e0e431e9fa8b17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:19:35.605218 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-00210e09-4558-4c4b-9d42-e33b2ba92b5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:35.606282 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-00210e09-4558-4c4b-9d42-e33b2ba92b5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:35.609070 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 899/1789] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:19:35 2022] DELETE /identity/v3/projects/8d3a17536478469e80e0e431e9fa8b17 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:35.610205 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00210e09-4558-4c4b-9d42-e33b2ba92b5b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:35.611792 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-00210e09-4558-4c4b-9d42-e33b2ba92b5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:35.660437 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:35.703423 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2398e738-267f-4730-b68f-1cb55800232e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:35.704277 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2398e738-267f-4730-b68f-1cb55800232e service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:35.704646 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2398e738-267f-4730-b68f-1cb55800232e service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:35.705009 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2398e738-267f-4730-b68f-1cb55800232e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:35.710193 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-2398e738-267f-4730-b68f-1cb55800232e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:35.722520 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2398e738-267f-4730-b68f-1cb55800232e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:35.724549 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2398e738-267f-4730-b68f-1cb55800232e service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:35.752625 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 891/1790] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:19:35.758330 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bbc039-f2a7-42dd-a243-38f05b44a191 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:35.758995 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bbc039-f2a7-42dd-a243-38f05b44a191 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:35.760305 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bbc039-f2a7-42dd-a243-38f05b44a191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:35.795134 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-f8bbc039-f2a7-42dd-a243-38f05b44a191 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:35.796389 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f8bbc039-f2a7-42dd-a243-38f05b44a191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:35.844628 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00210e09-4558-4c4b-9d42-e33b2ba92b5b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:35.874754 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21645f7f-cb14-42ab-93ef-b11f27055f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:35.876602 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21645f7f-cb14-42ab-93ef-b11f27055f02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:35.876992 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21645f7f-cb14-42ab-93ef-b11f27055f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:35.877384 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21645f7f-cb14-42ab-93ef-b11f27055f02 admin admin] PATH_INFO: `/v3/projects/1ac1170c339f4d5fa4a9ce7cbbdc21e4/users/7af5a088140e4809bb7cc9aa8060fd6d/roles/f36e053b442540f08041ac7878760f65` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:35.902185 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21645f7f-cb14-42ab-93ef-b11f27055f02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ac1170c339f4d5fa4a9ce7cbbdc21e4, user_id=7af5a088140e4809bb7cc9aa8060fd6d, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:35.915177 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21645f7f-cb14-42ab-93ef-b11f27055f02 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:35.938173 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 892/1791] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:35.939090 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bbc039-f2a7-42dd-a243-38f05b44a191 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:35.940476 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f8bbc039-f2a7-42dd-a243-38f05b44a191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:35.965334 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 900/1792] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:19:35 2022] PUT /identity/v3/projects/1ac1170c339f4d5fa4a9ce7cbbdc21e4/users/7af5a088140e4809bb7cc9aa8060fd6d/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:35.973005 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21645f7f-cb14-42ab-93ef-b11f27055f02 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:35.973150 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bbc039-f2a7-42dd-a243-38f05b44a191 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:35.973257 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-21645f7f-cb14-42ab-93ef-b11f27055f02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:36.072036 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918f438e-2369-44c7-bfaa-9afbc9a08a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:36.072744 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f438e-2369-44c7-bfaa-9afbc9a08a1c admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:36.073057 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f438e-2369-44c7-bfaa-9afbc9a08a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:36.073348 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f438e-2369-44c7-bfaa-9afbc9a08a1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:36.074095 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918f438e-2369-44c7-bfaa-9afbc9a08a1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:36.075863 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918f438e-2369-44c7-bfaa-9afbc9a08a1c admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:36.093649 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 893/1793] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:35 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:19:36.095026 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0ccb7a-f518-4c07-acba-8a3b234425f4 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:36.095324 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0ccb7a-f518-4c07-acba-8a3b234425f4 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:36.095615 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0ccb7a-f518-4c07-acba-8a3b234425f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:36.137254 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-0b0ccb7a-f518-4c07-acba-8a3b234425f4 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:36.138519 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0b0ccb7a-f518-4c07-acba-8a3b234425f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:36.147320 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 894/1794] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:36.157170 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0faffd3-1415-4616-b1ad-76d63117ae46 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:36.157170 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0faffd3-1415-4616-b1ad-76d63117ae46 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:36.157170 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0faffd3-1415-4616-b1ad-76d63117ae46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:36.170027 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-b0faffd3-1415-4616-b1ad-76d63117ae46 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:36.171556 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b0faffd3-1415-4616-b1ad-76d63117ae46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:36.182874 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 895/1795] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:36.184339 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce021683-05fd-4244-b123-0841a868ebbe None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:36.184636 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce021683-05fd-4244-b123-0841a868ebbe None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:36.184926 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce021683-05fd-4244-b123-0841a868ebbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:36.185902 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21645f7f-cb14-42ab-93ef-b11f27055f02 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:36.211378 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:36.214692 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-ce021683-05fd-4244-b123-0841a868ebbe None None] MFA Rules not processed for user `7af5a088140e4809bb7cc9aa8060fd6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:36.215904 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ce021683-05fd-4244-b123-0841a868ebbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:36.228065 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b39b78-b841-417c-a826-7bfa4e05d117 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:36.229333 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b39b78-b841-417c-a826-7bfa4e05d117 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:36.229846 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b39b78-b841-417c-a826-7bfa4e05d117 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:36.230337 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b39b78-b841-417c-a826-7bfa4e05d117 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:36.261135 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b39b78-b841-417c-a826-7bfa4e05d117 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:36.269028 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b39b78-b841-417c-a826-7bfa4e05d117 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:36.302053 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 901/1796] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:19:36.307415 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b39b78-b841-417c-a826-7bfa4e05d117 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:36.363827 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b39b78-b841-417c-a826-7bfa4e05d117 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:36.385871 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2667b9-f38c-4e61-b159-0a4ae5ff3500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:36.387288 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2667b9-f38c-4e61-b159-0a4ae5ff3500 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:36.387824 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2667b9-f38c-4e61-b159-0a4ae5ff3500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:36.388412 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2667b9-f38c-4e61-b159-0a4ae5ff3500 admin admin] PATH_INFO: `/v3/projects/5a4193e976ff42a1bfe19deaca893953/users/34b6f1f779ee49d380d28bfc41116ce5/roles/f36e053b442540f08041ac7878760f65` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:36.420301 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2667b9-f38c-4e61-b159-0a4ae5ff3500 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a4193e976ff42a1bfe19deaca893953, user_id=34b6f1f779ee49d380d28bfc41116ce5, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:36.423571 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2667b9-f38c-4e61-b159-0a4ae5ff3500 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:36.445769 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 902/1797] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:19:36 2022] PUT /identity/v3/projects/5a4193e976ff42a1bfe19deaca893953/users/34b6f1f779ee49d380d28bfc41116ce5/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:36.447825 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d2040-78af-4d24-b5dd-19c60e2448cc None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:36.448680 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d2040-78af-4d24-b5dd-19c60e2448cc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:36.449235 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d2040-78af-4d24-b5dd-19c60e2448cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:36.453078 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 896/1798] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:36 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:36.457313 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce021683-05fd-4244-b123-0841a868ebbe None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:36.480072 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1c8d2040-78af-4d24-b5dd-19c60e2448cc None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:36.480304 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce021683-05fd-4244-b123-0841a868ebbe None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:36.482706 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1c8d2040-78af-4d24-b5dd-19c60e2448cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:36.613564 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 903/1799] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:36.614944 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d2040-78af-4d24-b5dd-19c60e2448cc None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:36.636213 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d2040-78af-4d24-b5dd-19c60e2448cc None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:36.643270 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c2ed0-771c-42e4-9e20-98e9bc55acc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:36.644240 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c2ed0-771c-42e4-9e20-98e9bc55acc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:36.644635 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c2ed0-771c-42e4-9e20-98e9bc55acc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:36.645318 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c2ed0-771c-42e4-9e20-98e9bc55acc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:36.646313 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20c2ed0-771c-42e4-9e20-98e9bc55acc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:36.648788 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20c2ed0-771c-42e4-9e20-98e9bc55acc5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:36.668478 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 904/1800] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:36 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:19:36.669818 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c2ed0-771c-42e4-9e20-98e9bc55acc5 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:36.675449 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c20c2ed0-771c-42e4-9e20-98e9bc55acc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:36.678592 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d568599b-c8b8-4cfd-a162-dcfdf3125832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:36.680441 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d568599b-c8b8-4cfd-a162-dcfdf3125832 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:36.680890 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d568599b-c8b8-4cfd-a162-dcfdf3125832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:36.681892 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d568599b-c8b8-4cfd-a162-dcfdf3125832 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:36.682754 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d568599b-c8b8-4cfd-a162-dcfdf3125832 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:19:36.683073 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d568599b-c8b8-4cfd-a162-dcfdf3125832 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:36.711570 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d568599b-c8b8-4cfd-a162-dcfdf3125832 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:36.725706 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:36.760536 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 897/1801] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:19:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:19:36.765257 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d568599b-c8b8-4cfd-a162-dcfdf3125832 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:36.791524 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d568599b-c8b8-4cfd-a162-dcfdf3125832 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:36.802683 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d6488b-a9de-4024-9d48-9d0a9f13cc4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:36.803373 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d6488b-a9de-4024-9d48-9d0a9f13cc4a admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:36.803660 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d6488b-a9de-4024-9d48-9d0a9f13cc4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:36.803943 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d6488b-a9de-4024-9d48-9d0a9f13cc4a admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:36.812341 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d6488b-a9de-4024-9d48-9d0a9f13cc4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:19:36.812741 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d6488b-a9de-4024-9d48-9d0a9f13cc4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:36.814475 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d6488b-a9de-4024-9d48-9d0a9f13cc4a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:36.853001 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 898/1802] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:19:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:19:36.853959 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d6488b-a9de-4024-9d48-9d0a9f13cc4a admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:36.880168 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c2ed0-771c-42e4-9e20-98e9bc55acc5 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:36.884909 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d6488b-a9de-4024-9d48-9d0a9f13cc4a admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:36.889903 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd25cb3-4fbb-43eb-8dbf-f7349f59956b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:36.892306 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd25cb3-4fbb-43eb-8dbf-f7349f59956b service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:36.892306 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd25cb3-4fbb-43eb-8dbf-f7349f59956b service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:36.894164 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd25cb3-4fbb-43eb-8dbf-f7349f59956b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:36.895421 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c127b3-cf4b-489d-b319-600399043bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:36.896608 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c127b3-cf4b-489d-b319-600399043bbc admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:36.897159 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c127b3-cf4b-489d-b319-600399043bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:36.897619 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c127b3-cf4b-489d-b319-600399043bbc admin admin] PATH_INFO: `/v3/projects/5a4193e976ff42a1bfe19deaca893953/users/34b6f1f779ee49d380d28bfc41116ce5/roles/f36e053b442540f08041ac7878760f65` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:36.901587 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c127b3-cf4b-489d-b319-600399043bbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a4193e976ff42a1bfe19deaca893953, user_id=34b6f1f779ee49d380d28bfc41116ce5, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:36.916953 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c127b3-cf4b-489d-b319-600399043bbc admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:36.930390 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 899/1803] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:19:36 2022] PUT /identity/v3/projects/5a4193e976ff42a1bfe19deaca893953/users/34b6f1f779ee49d380d28bfc41116ce5/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:36.931501 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c127b3-cf4b-489d-b319-600399043bbc admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:37.075413 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd25cb3-4fbb-43eb-8dbf-f7349f59956b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:37.076202 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c127b3-cf4b-489d-b319-600399043bbc admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:37.077556 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd25cb3-4fbb-43eb-8dbf-f7349f59956b service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:37.082628 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46684b2e-3a6c-4ae9-a5c6-d75bc241d3b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:37.083541 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46684b2e-3a6c-4ae9-a5c6-d75bc241d3b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:37.083984 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46684b2e-3a6c-4ae9-a5c6-d75bc241d3b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:37.084322 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46684b2e-3a6c-4ae9-a5c6-d75bc241d3b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:37.085191 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46684b2e-3a6c-4ae9-a5c6-d75bc241d3b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:37.088280 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46684b2e-3a6c-4ae9-a5c6-d75bc241d3b1 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:37.104506 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 905/1804] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:36 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:19:37.105748 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:37.109022 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd25cb3-4fbb-43eb-8dbf-f7349f59956b service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:37.129370 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 900/1805] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:19:36 2022] POST /identity/v3/projects => generated 363 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:19:37.130999 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f5c82-d428-4095-952d-5a4c87ee18c5 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:37.131370 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f5c82-d428-4095-952d-5a4c87ee18c5 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:37.131600 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f5c82-d428-4095-952d-5a4c87ee18c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:37.144503 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd25cb3-4fbb-43eb-8dbf-f7349f59956b service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:37.233944 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-8d5f5c82-d428-4095-952d-5a4c87ee18c5 None None] MFA Rules not processed for user `34b6f1f779ee49d380d28bfc41116ce5`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:37.235314 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-8d5f5c82-d428-4095-952d-5a4c87ee18c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:37.285644 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cda463-cf8a-4b2e-9509-b9d026a13cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:37.287212 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cda463-cf8a-4b2e-9509-b9d026a13cc3 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:37.287711 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cda463-cf8a-4b2e-9509-b9d026a13cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:37.288224 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cda463-cf8a-4b2e-9509-b9d026a13cc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:37.289487 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cda463-cf8a-4b2e-9509-b9d026a13cc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:37.293603 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cda463-cf8a-4b2e-9509-b9d026a13cc3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:37.345259 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 906/1806] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:19:37 2022] POST /identity/v3/projects => generated 367 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:19:37.346756 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cda463-cf8a-4b2e-9509-b9d026a13cc3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:37.447990 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:37.462337 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 901/1807] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:37 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:37.463281 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f5c82-d428-4095-952d-5a4c87ee18c5 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:37.474844 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cda463-cf8a-4b2e-9509-b9d026a13cc3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:37.480835 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80030cd5-79b1-4d6b-a090-dee695f9b827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:37.481667 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80030cd5-79b1-4d6b-a090-dee695f9b827 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:37.482017 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80030cd5-79b1-4d6b-a090-dee695f9b827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:37.482364 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80030cd5-79b1-4d6b-a090-dee695f9b827 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:37.483138 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80030cd5-79b1-4d6b-a090-dee695f9b827 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:37.486135 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80030cd5-79b1-4d6b-a090-dee695f9b827 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:37.512908 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 907/1808] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:19:37 2022] POST /identity/v3/users => generated 282 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:19:37.514388 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80030cd5-79b1-4d6b-a090-dee695f9b827 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:37.530277 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80030cd5-79b1-4d6b-a090-dee695f9b827 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:37.536243 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d490f5d5-597b-4579-822e-ab396f19ebd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:37.536869 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490f5d5-597b-4579-822e-ab396f19ebd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:37.537263 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490f5d5-597b-4579-822e-ab396f19ebd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:37.537635 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490f5d5-597b-4579-822e-ab396f19ebd0 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:37.538275 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d490f5d5-597b-4579-822e-ab396f19ebd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:37.541267 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d490f5d5-597b-4579-822e-ab396f19ebd0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:37.565571 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 908/1809] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:19:37 2022] POST /identity/v3/users => generated 284 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:19:37.567469 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d490f5d5-597b-4579-822e-ab396f19ebd0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:37.570703 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d490f5d5-597b-4579-822e-ab396f19ebd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:37.629328 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f5c82-d428-4095-952d-5a4c87ee18c5 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:37.659992 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d490f5d5-597b-4579-822e-ab396f19ebd0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:37.670894 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cd288e-5d4e-4056-bcf7-93517c299aba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:37.672955 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cd288e-5d4e-4056-bcf7-93517c299aba service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:37.673619 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cd288e-5d4e-4056-bcf7-93517c299aba service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:37.674224 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cd288e-5d4e-4056-bcf7-93517c299aba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:37.710630 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943074c0-7129-45f3-aa5e-70e8f4fdb633 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:37.711558 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943074c0-7129-45f3-aa5e-70e8f4fdb633 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:37.713326 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943074c0-7129-45f3-aa5e-70e8f4fdb633 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:37.714868 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943074c0-7129-45f3-aa5e-70e8f4fdb633 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:37.718023 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-943074c0-7129-45f3-aa5e-70e8f4fdb633 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:37.781465 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cd288e-5d4e-4056-bcf7-93517c299aba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:37.784029 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:37.786425 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cd288e-5d4e-4056-bcf7-93517c299aba service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:37.805022 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 909/1810] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:37 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:19:37.806943 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cd288e-5d4e-4056-bcf7-93517c299aba service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:37.811301 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943074c0-7129-45f3-aa5e-70e8f4fdb633 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:37.812754 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943074c0-7129-45f3-aa5e-70e8f4fdb633 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:37.822552 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cd288e-5d4e-4056-bcf7-93517c299aba service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:37.828061 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452cd522-66c6-420a-ae5b-cbf7d2b42771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:37.829011 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452cd522-66c6-420a-ae5b-cbf7d2b42771 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:37.829514 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452cd522-66c6-420a-ae5b-cbf7d2b42771 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:37.829975 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452cd522-66c6-420a-ae5b-cbf7d2b42771 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:37.831033 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452cd522-66c6-420a-ae5b-cbf7d2b42771 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:37.832636 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 902/1811] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:37 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:19:37.833935 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452cd522-66c6-420a-ae5b-cbf7d2b42771 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:37.848688 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 910/1812] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:37 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:19:37.854648 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452cd522-66c6-420a-ae5b-cbf7d2b42771 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:37.855520 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943074c0-7129-45f3-aa5e-70e8f4fdb633 service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:37.873751 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452cd522-66c6-420a-ae5b-cbf7d2b42771 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:37.879790 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0302e82-3565-4883-8382-b17481bcda2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:37.880734 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0302e82-3565-4883-8382-b17481bcda2a admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:37.881150 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0302e82-3565-4883-8382-b17481bcda2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:37.881679 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0302e82-3565-4883-8382-b17481bcda2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:37.883497 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0302e82-3565-4883-8382-b17481bcda2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:37.885216 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0302e82-3565-4883-8382-b17481bcda2a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:37.889289 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943074c0-7129-45f3-aa5e-70e8f4fdb633 service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:37.898221 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 911/1813] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:37 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:19:37.902344 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0302e82-3565-4883-8382-b17481bcda2a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:37.906197 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08187b62-7640-444c-99ca-2e313cd7463f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:37.907243 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08187b62-7640-444c-99ca-2e313cd7463f admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:37.907730 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08187b62-7640-444c-99ca-2e313cd7463f admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:37.908249 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08187b62-7640-444c-99ca-2e313cd7463f admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:37.909443 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08187b62-7640-444c-99ca-2e313cd7463f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:37.914732 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08187b62-7640-444c-99ca-2e313cd7463f admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:37.926507 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0302e82-3565-4883-8382-b17481bcda2a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:37.944079 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bb6eee-5ec0-4b71-8c91-c3a2f868beb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:37.944788 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 903/1814] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:37 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:19:37.946328 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb6eee-5ec0-4b71-8c91-c3a2f868beb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:37.947783 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb6eee-5ec0-4b71-8c91-c3a2f868beb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:37.948318 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb6eee-5ec0-4b71-8c91-c3a2f868beb7 admin admin] PATH_INFO: `/v3/projects/45f92a6dd6eb44c5bf9d20337ecfdb76/users/43e5acd365404c34aa55591cae638c05/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:37.948719 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08187b62-7640-444c-99ca-2e313cd7463f admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:37.976749 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08187b62-7640-444c-99ca-2e313cd7463f admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:37.985154 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bb6eee-5ec0-4b71-8c91-c3a2f868beb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45f92a6dd6eb44c5bf9d20337ecfdb76, user_id=43e5acd365404c34aa55591cae638c05, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:37.989211 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bb6eee-5ec0-4b71-8c91-c3a2f868beb7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:37.992434 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca14283-8a43-4c05-b233-1eb9cc33c5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:37.993707 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca14283-8a43-4c05-b233-1eb9cc33c5a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:37.994288 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca14283-8a43-4c05-b233-1eb9cc33c5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:37.994745 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca14283-8a43-4c05-b233-1eb9cc33c5a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:38.001945 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca14283-8a43-4c05-b233-1eb9cc33c5a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:38.004149 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca14283-8a43-4c05-b233-1eb9cc33c5a4 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:38.015651 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 912/1815] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:19:37 2022] PUT /identity/v3/projects/45f92a6dd6eb44c5bf9d20337ecfdb76/users/43e5acd365404c34aa55591cae638c05/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:38.017845 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bb6eee-5ec0-4b71-8c91-c3a2f868beb7 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:38.081413 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 904/1816] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:37 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:19:38.082877 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5707284-baa0-4bc3-adc6-b4bbcc0e9f11 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:38.083175 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5707284-baa0-4bc3-adc6-b4bbcc0e9f11 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:38.083462 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5707284-baa0-4bc3-adc6-b4bbcc0e9f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:38.085894 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:38.155256 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-f5707284-baa0-4bc3-adc6-b4bbcc0e9f11 None None] MFA Rules not processed for user `43e5acd365404c34aa55591cae638c05`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:38.157633 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f5707284-baa0-4bc3-adc6-b4bbcc0e9f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:38.172663 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bb6eee-5ec0-4b71-8c91-c3a2f868beb7 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:38.182909 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0cde04-ba16-4deb-a3f5-73d45d9df553 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:38.185460 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0cde04-ba16-4deb-a3f5-73d45d9df553 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:38.187387 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0cde04-ba16-4deb-a3f5-73d45d9df553 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:38.187685 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0cde04-ba16-4deb-a3f5-73d45d9df553 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:38.193839 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ed0cde04-ba16-4deb-a3f5-73d45d9df553 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:38.354246 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 905/1817] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:38 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:38.355196 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5707284-baa0-4bc3-adc6-b4bbcc0e9f11 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:38.381611 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0cde04-ba16-4deb-a3f5-73d45d9df553 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:38.398929 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0cde04-ba16-4deb-a3f5-73d45d9df553 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:38.443893 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 913/1818] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:38 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:19:38.449314 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0cde04-ba16-4deb-a3f5-73d45d9df553 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:38.562319 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5707284-baa0-4bc3-adc6-b4bbcc0e9f11 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:38.576851 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8424fe9-e01a-4ae7-9cae-b8004ab60416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:38.580489 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8424fe9-e01a-4ae7-9cae-b8004ab60416 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:38.580769 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8424fe9-e01a-4ae7-9cae-b8004ab60416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:38.582580 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8424fe9-e01a-4ae7-9cae-b8004ab60416 admin admin] PATH_INFO: `/v3/projects/ba522c5410c84b8583d9d766309483cc/users/a98f8c510ce3485a99298d4d904f8b1d/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:38.692701 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8424fe9-e01a-4ae7-9cae-b8004ab60416 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba522c5410c84b8583d9d766309483cc, user_id=a98f8c510ce3485a99298d4d904f8b1d, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:38.719527 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8424fe9-e01a-4ae7-9cae-b8004ab60416 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:38.789406 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 906/1819] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:19:38 2022] PUT /identity/v3/projects/ba522c5410c84b8583d9d766309483cc/users/a98f8c510ce3485a99298d4d904f8b1d/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:38.794489 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45186a8-e1a3-492d-9b96-9e8e95801f38 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:38.794807 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45186a8-e1a3-492d-9b96-9e8e95801f38 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:38.797067 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45186a8-e1a3-492d-9b96-9e8e95801f38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:38.814527 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0cde04-ba16-4deb-a3f5-73d45d9df553 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:38.871314 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-f45186a8-e1a3-492d-9b96-9e8e95801f38 None None] MFA Rules not processed for user `a98f8c510ce3485a99298d4d904f8b1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:38.872628 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f45186a8-e1a3-492d-9b96-9e8e95801f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:38.932007 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:38.989933 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a1f6da-2d64-4b51-8a26-ff9299aa8e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:38.990651 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1f6da-2d64-4b51-8a26-ff9299aa8e59 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:38.990944 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1f6da-2d64-4b51-8a26-ff9299aa8e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:38.991226 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1f6da-2d64-4b51-8a26-ff9299aa8e59 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:38.992594 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a1f6da-2d64-4b51-8a26-ff9299aa8e59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:38.994681 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a1f6da-2d64-4b51-8a26-ff9299aa8e59 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:39.061220 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 907/1820] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:38 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:39.062386 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45186a8-e1a3-492d-9b96-9e8e95801f38 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:39.082537 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 914/1821] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:19:38 2022] POST /identity/v3/projects => generated 365 bytes in 638 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:19:39.083657 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a1f6da-2d64-4b51-8a26-ff9299aa8e59 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:39.090221 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45186a8-e1a3-492d-9b96-9e8e95801f38 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:39.125401 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:39.212950 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9363e821-04e9-4372-b38d-4ea994e5064b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:39.213970 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9363e821-04e9-4372-b38d-4ea994e5064b admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:39.214264 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9363e821-04e9-4372-b38d-4ea994e5064b admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:39.214553 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9363e821-04e9-4372-b38d-4ea994e5064b admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:39.215324 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9363e821-04e9-4372-b38d-4ea994e5064b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:39.220595 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9363e821-04e9-4372-b38d-4ea994e5064b admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:39.228221 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a1f6da-2d64-4b51-8a26-ff9299aa8e59 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:39.257750 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18244c53-22b9-47d6-ba8b-fdc0298d57d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:39.259707 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18244c53-22b9-47d6-ba8b-fdc0298d57d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:39.260001 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18244c53-22b9-47d6-ba8b-fdc0298d57d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:39.260282 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18244c53-22b9-47d6-ba8b-fdc0298d57d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:39.263563 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18244c53-22b9-47d6-ba8b-fdc0298d57d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:39.270518 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18244c53-22b9-47d6-ba8b-fdc0298d57d4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:39.306488 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 908/1822] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:19:39 2022] POST /identity/v3/projects => generated 367 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:19:39.311017 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9363e821-04e9-4372-b38d-4ea994e5064b admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:39.338537 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 915/1823] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:19:39 2022] POST /identity/v3/users => generated 289 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:19:39.341418 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18244c53-22b9-47d6-ba8b-fdc0298d57d4 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:39.370778 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18244c53-22b9-47d6-ba8b-fdc0298d57d4 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:39.424716 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9363e821-04e9-4372-b38d-4ea994e5064b admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:39.432936 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ad9f26-be5e-4a00-86ae-c243b1c53d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:39.433687 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad9f26-be5e-4a00-86ae-c243b1c53d59 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:39.434207 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad9f26-be5e-4a00-86ae-c243b1c53d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:39.434502 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad9f26-be5e-4a00-86ae-c243b1c53d59 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:39.435375 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ad9f26-be5e-4a00-86ae-c243b1c53d59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:39.437951 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ad9f26-be5e-4a00-86ae-c243b1c53d59 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:39.485355 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 909/1824] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:19:39 2022] POST /identity/v3/users => generated 290 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:19:39.486564 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ad9f26-be5e-4a00-86ae-c243b1c53d59 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:39.502134 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ad9f26-be5e-4a00-86ae-c243b1c53d59 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:39.512970 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f48dc4c-20ba-4558-a3b6-ce5fd2ceeb6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:39.513866 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f48dc4c-20ba-4558-a3b6-ce5fd2ceeb6a admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:39.514204 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f48dc4c-20ba-4558-a3b6-ce5fd2ceeb6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:39.514496 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f48dc4c-20ba-4558-a3b6-ce5fd2ceeb6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:39.515386 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f48dc4c-20ba-4558-a3b6-ce5fd2ceeb6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:39.517127 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f48dc4c-20ba-4558-a3b6-ce5fd2ceeb6a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:39.528701 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cc7a3d-9214-4132-9ef7-ed500c40cda5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:39.529752 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cc7a3d-9214-4132-9ef7-ed500c40cda5 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:39.530144 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cc7a3d-9214-4132-9ef7-ed500c40cda5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:39.530696 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cc7a3d-9214-4132-9ef7-ed500c40cda5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:39.531736 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cc7a3d-9214-4132-9ef7-ed500c40cda5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:39.533566 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cc7a3d-9214-4132-9ef7-ed500c40cda5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:39.534813 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 910/1825] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:39 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:19:39.542603 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f48dc4c-20ba-4558-a3b6-ce5fd2ceeb6a admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:39.549350 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 916/1826] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:39 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:19:39.550435 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cc7a3d-9214-4132-9ef7-ed500c40cda5 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:39.562278 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f48dc4c-20ba-4558-a3b6-ce5fd2ceeb6a admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:39.565064 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cc7a3d-9214-4132-9ef7-ed500c40cda5 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:39.567610 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae08d89-fbca-43e4-80db-40d837cff8b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:39.568720 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae08d89-fbca-43e4-80db-40d837cff8b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:39.568804 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae08d89-fbca-43e4-80db-40d837cff8b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:39.569167 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae08d89-fbca-43e4-80db-40d837cff8b7 admin admin] PATH_INFO: `/v3/projects/3b3aa61f57ee4e4b9cbb6da78de2c577/users/0cb6c64c64fa4caa94e9ec91d0eeb693/roles/f36e053b442540f08041ac7878760f65` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:39.571990 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad420e56-f4c1-44d6-9cea-8ddae5a9e783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:39.573473 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad420e56-f4c1-44d6-9cea-8ddae5a9e783 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:39.573977 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad420e56-f4c1-44d6-9cea-8ddae5a9e783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:39.574357 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad420e56-f4c1-44d6-9cea-8ddae5a9e783 admin admin] PATH_INFO: `/v3/projects/2ae26f7176544e4a9e1d96369edaa354/users/f136289ff5ff49a391e0f9e880a51cb1/roles/f36e053b442540f08041ac7878760f65` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:39.595199 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae08d89-fbca-43e4-80db-40d837cff8b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b3aa61f57ee4e4b9cbb6da78de2c577, user_id=0cb6c64c64fa4caa94e9ec91d0eeb693, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:39.596254 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad420e56-f4c1-44d6-9cea-8ddae5a9e783 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ae26f7176544e4a9e1d96369edaa354, user_id=f136289ff5ff49a391e0f9e880a51cb1, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:39.599828 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad420e56-f4c1-44d6-9cea-8ddae5a9e783 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:39.606173 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae08d89-fbca-43e4-80db-40d837cff8b7 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:39.621300 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 917/1827] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:19:39 2022] PUT /identity/v3/projects/2ae26f7176544e4a9e1d96369edaa354/users/f136289ff5ff49a391e0f9e880a51cb1/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:39.630346 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad420e56-f4c1-44d6-9cea-8ddae5a9e783 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:39.637754 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 911/1828] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:19:39 2022] PUT /identity/v3/projects/3b3aa61f57ee4e4b9cbb6da78de2c577/users/0cb6c64c64fa4caa94e9ec91d0eeb693/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:39.646850 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae08d89-fbca-43e4-80db-40d837cff8b7 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:39.673738 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:39.822779 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad420e56-f4c1-44d6-9cea-8ddae5a9e783 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:39.850285 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1a30a0-c899-4269-9f8a-1293fced5024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:39.851455 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a30a0-c899-4269-9f8a-1293fced5024 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:39.852082 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a30a0-c899-4269-9f8a-1293fced5024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:39.852746 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a30a0-c899-4269-9f8a-1293fced5024 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:39.853509 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1a30a0-c899-4269-9f8a-1293fced5024 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:39.855151 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1a30a0-c899-4269-9f8a-1293fced5024 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:39.862125 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae08d89-fbca-43e4-80db-40d837cff8b7 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:39.878257 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a17416-5bcb-49c9-9a8c-96848c6afd3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:39.885583 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a17416-5bcb-49c9-9a8c-96848c6afd3a admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:39.885583 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a17416-5bcb-49c9-9a8c-96848c6afd3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:39.885583 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a17416-5bcb-49c9-9a8c-96848c6afd3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:39.885583 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a17416-5bcb-49c9-9a8c-96848c6afd3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:39.885583 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a17416-5bcb-49c9-9a8c-96848c6afd3a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:39.897951 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 918/1829] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:39 2022] GET /identity/v3/roles => generated 1420 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:19:39.902284 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1a30a0-c899-4269-9f8a-1293fced5024 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:39.942341 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 912/1830] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:39 2022] GET /identity/v3/roles => generated 1420 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:19:39.943541 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a17416-5bcb-49c9-9a8c-96848c6afd3a admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:39.944434 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1a30a0-c899-4269-9f8a-1293fced5024 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:39.967128 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde4f628-d3f1-4678-9539-28ecfb7161ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:39.970863 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4f628-d3f1-4678-9539-28ecfb7161ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:39.971205 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4f628-d3f1-4678-9539-28ecfb7161ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:39.971485 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4f628-d3f1-4678-9539-28ecfb7161ca admin admin] PATH_INFO: `/v3/projects/2ae26f7176544e4a9e1d96369edaa354/users/f136289ff5ff49a391e0f9e880a51cb1/roles/f36e053b442540f08041ac7878760f65` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:39.978092 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a17416-5bcb-49c9-9a8c-96848c6afd3a admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:39.980917 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde4f628-d3f1-4678-9539-28ecfb7161ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ae26f7176544e4a9e1d96369edaa354, user_id=f136289ff5ff49a391e0f9e880a51cb1, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:39.988281 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde4f628-d3f1-4678-9539-28ecfb7161ca admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:39.999357 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b4f376-8f96-4480-ba1c-54e7e6ac2d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:40.002264 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b4f376-8f96-4480-ba1c-54e7e6ac2d60 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:40.004196 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b4f376-8f96-4480-ba1c-54e7e6ac2d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:40.004529 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b4f376-8f96-4480-ba1c-54e7e6ac2d60 admin admin] PATH_INFO: `/v3/projects/3b3aa61f57ee4e4b9cbb6da78de2c577/users/0cb6c64c64fa4caa94e9ec91d0eeb693/roles/f36e053b442540f08041ac7878760f65` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:40.017383 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b4f376-8f96-4480-ba1c-54e7e6ac2d60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b3aa61f57ee4e4b9cbb6da78de2c577, user_id=0cb6c64c64fa4caa94e9ec91d0eeb693, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:40.021543 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:40.026248 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b4f376-8f96-4480-ba1c-54e7e6ac2d60 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:40.058983 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 919/1831] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:19:39 2022] PUT /identity/v3/projects/2ae26f7176544e4a9e1d96369edaa354/users/f136289ff5ff49a391e0f9e880a51cb1/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:40.060879 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ee355-e872-4b2e-9166-32768ce719b9 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:40.061203 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ee355-e872-4b2e-9166-32768ce719b9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:40.061490 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ee355-e872-4b2e-9166-32768ce719b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:40.072171 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 913/1832] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:19:39 2022] PUT /identity/v3/projects/3b3aa61f57ee4e4b9cbb6da78de2c577/users/0cb6c64c64fa4caa94e9ec91d0eeb693/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:40.085488 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d64710-ef77-44bb-aac0-688075f0efa0 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:40.087143 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d64710-ef77-44bb-aac0-688075f0efa0 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:40.087440 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d64710-ef77-44bb-aac0-688075f0efa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:40.169716 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3f4ee355-e872-4b2e-9166-32768ce719b9 None None] MFA Rules not processed for user `f136289ff5ff49a391e0f9e880a51cb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:40.176717 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3f4ee355-e872-4b2e-9166-32768ce719b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:40.208262 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-f1d64710-ef77-44bb-aac0-688075f0efa0 None None] MFA Rules not processed for user `0cb6c64c64fa4caa94e9ec91d0eeb693`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:40.209584 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f1d64710-ef77-44bb-aac0-688075f0efa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:40.444468 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 920/1833] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:40 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 385 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:40.472290 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 914/1834] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:40 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 388 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:19:40.517542 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ee355-e872-4b2e-9166-32768ce719b9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:40.582240 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d64710-ef77-44bb-aac0-688075f0efa0 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:40.623089 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ee355-e872-4b2e-9166-32768ce719b9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:40.628075 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ff3d3-055a-4da7-893c-a0310f47b8f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:40.628942 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ff3d3-055a-4da7-893c-a0310f47b8f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:40.629342 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ff3d3-055a-4da7-893c-a0310f47b8f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:40.630571 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ff3d3-055a-4da7-893c-a0310f47b8f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:40.650248 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6ff3d3-055a-4da7-893c-a0310f47b8f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:40.651870 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6ff3d3-055a-4da7-893c-a0310f47b8f8 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:40.669375 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 921/1835] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:40 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:19:40.764777 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d64710-ef77-44bb-aac0-688075f0efa0 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:40.780047 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87caafd5-5b43-4ab4-9616-d50485bbe1d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:40.780833 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87caafd5-5b43-4ab4-9616-d50485bbe1d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:40.781283 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87caafd5-5b43-4ab4-9616-d50485bbe1d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:40.781595 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87caafd5-5b43-4ab4-9616-d50485bbe1d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:40.799283 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87caafd5-5b43-4ab4-9616-d50485bbe1d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:40.800926 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87caafd5-5b43-4ab4-9616-d50485bbe1d6 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:40.828415 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 915/1836] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:40 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:19:41.080786 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ff3d3-055a-4da7-893c-a0310f47b8f8 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:41.095494 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ff3d3-055a-4da7-893c-a0310f47b8f8 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:41.126586 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0c2e9b-c732-404b-b415-a2326a410332 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:41.127645 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0c2e9b-c732-404b-b415-a2326a410332 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:41.128079 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0c2e9b-c732-404b-b415-a2326a410332 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:41.129401 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0c2e9b-c732-404b-b415-a2326a410332 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:41.190929 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7866a-942a-4aee-9aff-7634641aa15d None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:41.191407 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7866a-942a-4aee-9aff-7634641aa15d None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:41.192199 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7866a-942a-4aee-9aff-7634641aa15d None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:41.199745 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 916/1837] 10.222.0.71 () {54 vars in 921 bytes} [Mon Jul 4 09:19:41 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:19:41.219802 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e09f8b-042e-4549-b7c9-ba4714695607 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:41.220270 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e09f8b-042e-4549-b7c9-ba4714695607 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:41.220667 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e09f8b-042e-4549-b7c9-ba4714695607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:41.325455 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-f1e09f8b-042e-4549-b7c9-ba4714695607 None None] MFA Rules not processed for user `15039da209ec445d93d49e2bd6d854eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:41.334392 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f1e09f8b-042e-4549-b7c9-ba4714695607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:41.386219 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0c2e9b-c732-404b-b415-a2326a410332 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:41.388127 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0c2e9b-c732-404b-b415-a2326a410332 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:41.412340 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:41.416504 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 922/1838] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:41 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:19:41.673172 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 917/1839] 10.222.0.71 () {58 vars in 1018 bytes} [Mon Jul 4 09:19:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 454 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jul 04 09:19:42.369286 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0c2e9b-c732-404b-b415-a2326a410332 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:42.372413 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4e0c2e9b-c732-404b-b415-a2326a410332 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:42.411896 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0c2e9b-c732-404b-b415-a2326a410332 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:42.425181 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e2db97-00a8-47c5-81de-46da13e97680 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15039da209ec445d93d49e2bd6d854eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:42.425859 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e2db97-00a8-47c5-81de-46da13e97680 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:42.426142 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e2db97-00a8-47c5-81de-46da13e97680 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:42.428722 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e2db97-00a8-47c5-81de-46da13e97680 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:42.440311 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e09f8b-042e-4549-b7c9-ba4714695607 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:42.451669 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e2db97-00a8-47c5-81de-46da13e97680 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:42.454266 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e2db97-00a8-47c5-81de-46da13e97680 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:42.483067 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 923/1840] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:19:42.532103 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e09f8b-042e-4549-b7c9-ba4714695607 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:42.539742 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092f1125-07ad-4048-8f6e-107f464c374e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:42.540800 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f1125-07ad-4048-8f6e-107f464c374e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:42.541232 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f1125-07ad-4048-8f6e-107f464c374e admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:42.541623 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f1125-07ad-4048-8f6e-107f464c374e admin admin] PATH_INFO: `/v3/users/f3864d1503914ca0a6140cedabee06db` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:42.543351 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092f1125-07ad-4048-8f6e-107f464c374e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3864d1503914ca0a6140cedabee06db)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:42.545847 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092f1125-07ad-4048-8f6e-107f464c374e admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:42.650679 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e2db97-00a8-47c5-81de-46da13e97680 service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:42.663974 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e2db97-00a8-47c5-81de-46da13e97680 service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:42.669191 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee2639a-428e-4bd8-8089-9568347e7a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:42.670262 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee2639a-428e-4bd8-8089-9568347e7a6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:42.670605 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee2639a-428e-4bd8-8089-9568347e7a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:42.671058 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee2639a-428e-4bd8-8089-9568347e7a6a admin admin] PATH_INFO: `/v3/users/43e5acd365404c34aa55591cae638c05` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:42.672765 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee2639a-428e-4bd8-8089-9568347e7a6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=43e5acd365404c34aa55591cae638c05)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:42.674748 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee2639a-428e-4bd8-8089-9568347e7a6a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:42.684119 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:42.732928 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-092f1125-07ad-4048-8f6e-107f464c374e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3864d1503914ca0a6140cedabee06db'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:42.743067 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-092f1125-07ad-4048-8f6e-107f464c374e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3864d1503914ca0a6140cedabee06db'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:42.743825 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-092f1125-07ad-4048-8f6e-107f464c374e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3864d1503914ca0a6140cedabee06db'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:42.748019 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8ee2639a-428e-4bd8-8089-9568347e7a6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43e5acd365404c34aa55591cae638c05'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:42.748684 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8ee2639a-428e-4bd8-8089-9568347e7a6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43e5acd365404c34aa55591cae638c05'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:42.753704 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-092f1125-07ad-4048-8f6e-107f464c374e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3864d1503914ca0a6140cedabee06db'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:42.758698 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8ee2639a-428e-4bd8-8089-9568347e7a6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43e5acd365404c34aa55591cae638c05'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:42.771614 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-092f1125-07ad-4048-8f6e-107f464c374e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3864d1503914ca0a6140cedabee06db'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:42.779297 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8ee2639a-428e-4bd8-8089-9568347e7a6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43e5acd365404c34aa55591cae638c05'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:42.780085 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 918/1841] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:19:42 2022] DELETE /identity/v3/users/f3864d1503914ca0a6140cedabee06db => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:19:42.780260 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8ee2639a-428e-4bd8-8089-9568347e7a6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43e5acd365404c34aa55591cae638c05'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:42.785329 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774d0688-987f-4298-8a1e-6f2d42d3772d None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:42.785921 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774d0688-987f-4298-8a1e-6f2d42d3772d None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:42.786574 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774d0688-987f-4298-8a1e-6f2d42d3772d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:42.797820 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 924/1842] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:19:42 2022] DELETE /identity/v3/users/43e5acd365404c34aa55591cae638c05 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:19:42.802201 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81abeb02-d4a1-40ce-a066-92815f6e8a5e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:42.802493 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81abeb02-d4a1-40ce-a066-92815f6e8a5e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:42.802774 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81abeb02-d4a1-40ce-a066-92815f6e8a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:42.808752 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-774d0688-987f-4298-8a1e-6f2d42d3772d None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:42.810066 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-774d0688-987f-4298-8a1e-6f2d42d3772d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:42.822335 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-81abeb02-d4a1-40ce-a066-92815f6e8a5e None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:42.823813 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-81abeb02-d4a1-40ce-a066-92815f6e8a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:42.837371 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:42.884460 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 919/1843] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:42.893724 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774d0688-987f-4298-8a1e-6f2d42d3772d None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:42.895345 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-774d0688-987f-4298-8a1e-6f2d42d3772d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:42.899572 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 925/1844] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:42.908633 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81abeb02-d4a1-40ce-a066-92815f6e8a5e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:42.910025 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-81abeb02-d4a1-40ce-a066-92815f6e8a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:42.968049 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774d0688-987f-4298-8a1e-6f2d42d3772d None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:42.976762 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e901864-d59d-4d7c-8253-0e95dc68cc1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:42.977757 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e901864-d59d-4d7c-8253-0e95dc68cc1d service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:42.978120 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e901864-d59d-4d7c-8253-0e95dc68cc1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:42.978369 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81abeb02-d4a1-40ce-a066-92815f6e8a5e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:42.978486 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e901864-d59d-4d7c-8253-0e95dc68cc1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:42.982499 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ea51c0-454f-44b3-bbfa-bfcf54499e90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:42.983194 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea51c0-454f-44b3-bbfa-bfcf54499e90 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:42.983463 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea51c0-454f-44b3-bbfa-bfcf54499e90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:42.983735 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea51c0-454f-44b3-bbfa-bfcf54499e90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:42.993791 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e901864-d59d-4d7c-8253-0e95dc68cc1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:42.995285 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e901864-d59d-4d7c-8253-0e95dc68cc1d service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:42.996041 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ea51c0-454f-44b3-bbfa-bfcf54499e90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:42.997535 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ea51c0-454f-44b3-bbfa-bfcf54499e90 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:43.009221 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 920/1845] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:19:43.010503 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 926/1846] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:19:43.449578 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e901864-d59d-4d7c-8253-0e95dc68cc1d service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:43.454219 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-1e901864-d59d-4d7c-8253-0e95dc68cc1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:43.521616 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e901864-d59d-4d7c-8253-0e95dc68cc1d service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:43.542090 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a5bdbd-cc0d-4bfa-a9e3-5508f2845ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:43.546709 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a5bdbd-cc0d-4bfa-a9e3-5508f2845ad9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:43.547009 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a5bdbd-cc0d-4bfa-a9e3-5508f2845ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:43.547299 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a5bdbd-cc0d-4bfa-a9e3-5508f2845ad9 admin admin] PATH_INFO: `/v3/users/38842d0848054b2dbd14fe07dd0fef97` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:43.555708 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a5bdbd-cc0d-4bfa-a9e3-5508f2845ad9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38842d0848054b2dbd14fe07dd0fef97)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:43.562062 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a5bdbd-cc0d-4bfa-a9e3-5508f2845ad9 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:43.693791 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-79a5bdbd-cc0d-4bfa-a9e3-5508f2845ad9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38842d0848054b2dbd14fe07dd0fef97'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:43.716824 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-79a5bdbd-cc0d-4bfa-a9e3-5508f2845ad9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38842d0848054b2dbd14fe07dd0fef97'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:43.726980 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-79a5bdbd-cc0d-4bfa-a9e3-5508f2845ad9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38842d0848054b2dbd14fe07dd0fef97'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:43.737775 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-79a5bdbd-cc0d-4bfa-a9e3-5508f2845ad9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38842d0848054b2dbd14fe07dd0fef97'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:43.759786 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-79a5bdbd-cc0d-4bfa-a9e3-5508f2845ad9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38842d0848054b2dbd14fe07dd0fef97'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:43.780254 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7abed65-3201-49d0-a055-ad22c2a05692 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:43.780811 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 921/1847] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:19:43 2022] DELETE /identity/v3/users/38842d0848054b2dbd14fe07dd0fef97 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:43.781274 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7abed65-3201-49d0-a055-ad22c2a05692 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:43.781562 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7abed65-3201-49d0-a055-ad22c2a05692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:43.836150 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d7abed65-3201-49d0-a055-ad22c2a05692 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:43.837921 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d7abed65-3201-49d0-a055-ad22c2a05692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:43.903131 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:43.948263 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 927/1848] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:19:43.983648 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a5bdbd-cc0d-4bfa-a9e3-5508f2845ad9 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:43.985558 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-79a5bdbd-cc0d-4bfa-a9e3-5508f2845ad9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:44.091116 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:44.105730 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a5bdbd-cc0d-4bfa-a9e3-5508f2845ad9 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:44.113142 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b851e49-6ccf-4e8a-ac46-1051ebbd5d7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:44.114408 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b851e49-6ccf-4e8a-ac46-1051ebbd5d7a service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:44.115181 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b851e49-6ccf-4e8a-ac46-1051ebbd5d7a service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:44.115468 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b851e49-6ccf-4e8a-ac46-1051ebbd5d7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:44.130640 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b851e49-6ccf-4e8a-ac46-1051ebbd5d7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:44.132495 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b851e49-6ccf-4e8a-ac46-1051ebbd5d7a service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:44.145408 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 922/1849] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:19:44.258983 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7abed65-3201-49d0-a055-ad22c2a05692 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:44.260681 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d7abed65-3201-49d0-a055-ad22c2a05692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:44.286063 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7abed65-3201-49d0-a055-ad22c2a05692 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:44.292571 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9cf3d2-eb5c-4a16-a535-691d7ef0311b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:44.293645 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9cf3d2-eb5c-4a16-a535-691d7ef0311b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:44.293939 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9cf3d2-eb5c-4a16-a535-691d7ef0311b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:44.294938 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9cf3d2-eb5c-4a16-a535-691d7ef0311b admin admin] PATH_INFO: `/v3/projects/45f92a6dd6eb44c5bf9d20337ecfdb76` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:44.296203 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9cf3d2-eb5c-4a16-a535-691d7ef0311b admin admin] RBAC: Authorizing `identity:delete_project(project_id=45f92a6dd6eb44c5bf9d20337ecfdb76)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:44.298835 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9cf3d2-eb5c-4a16-a535-691d7ef0311b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:44.305024 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b851e49-6ccf-4e8a-ac46-1051ebbd5d7a service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:44.306758 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4b851e49-6ccf-4e8a-ac46-1051ebbd5d7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:44.327153 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b851e49-6ccf-4e8a-ac46-1051ebbd5d7a service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:44.362201 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fc1af5-71fe-4559-b7aa-b0551a5e197c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:44.363672 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc1af5-71fe-4559-b7aa-b0551a5e197c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:44.364108 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc1af5-71fe-4559-b7aa-b0551a5e197c admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:44.364784 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc1af5-71fe-4559-b7aa-b0551a5e197c admin admin] PATH_INFO: `/v3/projects/8a46c06e19854d13a197aa13cc2a75f4` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:44.366733 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fc1af5-71fe-4559-b7aa-b0551a5e197c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a46c06e19854d13a197aa13cc2a75f4)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:44.369114 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fc1af5-71fe-4559-b7aa-b0551a5e197c admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:44.400121 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3f9cf3d2-eb5c-4a16-a535-691d7ef0311b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45f92a6dd6eb44c5bf9d20337ecfdb76', 'request_id': 'req-3f9cf3d2-eb5c-4a16-a535-691d7ef0311b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:44.406538 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3f9cf3d2-eb5c-4a16-a535-691d7ef0311b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45f92a6dd6eb44c5bf9d20337ecfdb76', 'request_id': 'req-3f9cf3d2-eb5c-4a16-a535-691d7ef0311b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:44.413691 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3f9cf3d2-eb5c-4a16-a535-691d7ef0311b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45f92a6dd6eb44c5bf9d20337ecfdb76', 'request_id': 'req-3f9cf3d2-eb5c-4a16-a535-691d7ef0311b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:44.432592 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3f9cf3d2-eb5c-4a16-a535-691d7ef0311b admin admin] The token cache is being invalidate because project 45f92a6dd6eb44c5bf9d20337ecfdb76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:19:44.432901 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3f9cf3d2-eb5c-4a16-a535-691d7ef0311b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:44.433640 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3f9cf3d2-eb5c-4a16-a535-691d7ef0311b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:44.438097 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 928/1850] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:19:44 2022] DELETE /identity/v3/projects/45f92a6dd6eb44c5bf9d20337ecfdb76 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:44.439797 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9cf3d2-eb5c-4a16-a535-691d7ef0311b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:44.441185 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3f9cf3d2-eb5c-4a16-a535-691d7ef0311b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:44.478088 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f9fc1af5-71fe-4559-b7aa-b0551a5e197c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a46c06e19854d13a197aa13cc2a75f4', 'request_id': 'req-f9fc1af5-71fe-4559-b7aa-b0551a5e197c'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:44.486090 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f9fc1af5-71fe-4559-b7aa-b0551a5e197c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a46c06e19854d13a197aa13cc2a75f4', 'request_id': 'req-f9fc1af5-71fe-4559-b7aa-b0551a5e197c'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:44.497841 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f9fc1af5-71fe-4559-b7aa-b0551a5e197c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a46c06e19854d13a197aa13cc2a75f4', 'request_id': 'req-f9fc1af5-71fe-4559-b7aa-b0551a5e197c'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:44.524829 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f9fc1af5-71fe-4559-b7aa-b0551a5e197c admin admin] The token cache is being invalidate because project 8a46c06e19854d13a197aa13cc2a75f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:19:44.525601 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f9fc1af5-71fe-4559-b7aa-b0551a5e197c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:44.526219 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f9fc1af5-71fe-4559-b7aa-b0551a5e197c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:44.530137 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 923/1851] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:19:44 2022] DELETE /identity/v3/projects/8a46c06e19854d13a197aa13cc2a75f4 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:19:44.533893 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fc1af5-71fe-4559-b7aa-b0551a5e197c admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:44.535733 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f9fc1af5-71fe-4559-b7aa-b0551a5e197c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:44.577548 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9cf3d2-eb5c-4a16-a535-691d7ef0311b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:44.659633 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fc1af5-71fe-4559-b7aa-b0551a5e197c admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:44.665366 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd3201e-b0de-4289-91a4-375e4b90e9c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:44.666631 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3201e-b0de-4289-91a4-375e4b90e9c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:44.667060 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3201e-b0de-4289-91a4-375e4b90e9c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:44.667416 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3201e-b0de-4289-91a4-375e4b90e9c5 admin admin] PATH_INFO: `/v3/users/34b6f1f779ee49d380d28bfc41116ce5` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:44.667925 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47313c3-b2ac-4606-9c2e-85c4d3692192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:44.668986 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47313c3-b2ac-4606-9c2e-85c4d3692192 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:44.669268 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd3201e-b0de-4289-91a4-375e4b90e9c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34b6f1f779ee49d380d28bfc41116ce5)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:44.669628 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47313c3-b2ac-4606-9c2e-85c4d3692192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:44.670581 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47313c3-b2ac-4606-9c2e-85c4d3692192 admin admin] PATH_INFO: `/v3/users/f136289ff5ff49a391e0f9e880a51cb1` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:44.671525 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd3201e-b0de-4289-91a4-375e4b90e9c5 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:44.672162 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47313c3-b2ac-4606-9c2e-85c4d3692192 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f136289ff5ff49a391e0f9e880a51cb1)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:44.674476 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47313c3-b2ac-4606-9c2e-85c4d3692192 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:44.676725 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:44.758838 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e47313c3-b2ac-4606-9c2e-85c4d3692192 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f136289ff5ff49a391e0f9e880a51cb1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:44.759757 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e47313c3-b2ac-4606-9c2e-85c4d3692192 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f136289ff5ff49a391e0f9e880a51cb1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:44.761528 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-efd3201e-b0de-4289-91a4-375e4b90e9c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34b6f1f779ee49d380d28bfc41116ce5'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:44.774572 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e47313c3-b2ac-4606-9c2e-85c4d3692192 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f136289ff5ff49a391e0f9e880a51cb1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:44.786835 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-efd3201e-b0de-4289-91a4-375e4b90e9c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34b6f1f779ee49d380d28bfc41116ce5'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:44.786835 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-efd3201e-b0de-4289-91a4-375e4b90e9c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34b6f1f779ee49d380d28bfc41116ce5'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:44.802285 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-efd3201e-b0de-4289-91a4-375e4b90e9c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34b6f1f779ee49d380d28bfc41116ce5'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:44.804638 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e47313c3-b2ac-4606-9c2e-85c4d3692192 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f136289ff5ff49a391e0f9e880a51cb1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:44.806552 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e47313c3-b2ac-4606-9c2e-85c4d3692192 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f136289ff5ff49a391e0f9e880a51cb1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:44.815464 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:44.825889 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-efd3201e-b0de-4289-91a4-375e4b90e9c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34b6f1f779ee49d380d28bfc41116ce5'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:44.832276 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 929/1852] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:19:44 2022] DELETE /identity/v3/users/f136289ff5ff49a391e0f9e880a51cb1 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:44.832276 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47313c3-b2ac-4606-9c2e-85c4d3692192 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:44.836157 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e47313c3-b2ac-4606-9c2e-85c4d3692192 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:44.840956 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 924/1853] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:19:44 2022] DELETE /identity/v3/users/34b6f1f779ee49d380d28bfc41116ce5 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:44.943780 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47313c3-b2ac-4606-9c2e-85c4d3692192 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:45.035540 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1ebf48-7dbb-4e21-8419-ab298c7d11d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:45.037425 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ebf48-7dbb-4e21-8419-ab298c7d11d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:45.037948 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ebf48-7dbb-4e21-8419-ab298c7d11d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:45.038558 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ebf48-7dbb-4e21-8419-ab298c7d11d9 admin admin] PATH_INFO: `/v3/projects/f96f099eae0c4fe58ef30986514abbb8` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:45.040335 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1ebf48-7dbb-4e21-8419-ab298c7d11d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f96f099eae0c4fe58ef30986514abbb8)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:45.044973 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1ebf48-7dbb-4e21-8419-ab298c7d11d9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:45.116589 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:45.136139 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8e1ebf48-7dbb-4e21-8419-ab298c7d11d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f96f099eae0c4fe58ef30986514abbb8', 'request_id': 'req-8e1ebf48-7dbb-4e21-8419-ab298c7d11d9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:45.152296 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8e1ebf48-7dbb-4e21-8419-ab298c7d11d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f96f099eae0c4fe58ef30986514abbb8', 'request_id': 'req-8e1ebf48-7dbb-4e21-8419-ab298c7d11d9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:45.160933 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8e1ebf48-7dbb-4e21-8419-ab298c7d11d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f96f099eae0c4fe58ef30986514abbb8', 'request_id': 'req-8e1ebf48-7dbb-4e21-8419-ab298c7d11d9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:45.187192 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8e1ebf48-7dbb-4e21-8419-ab298c7d11d9 admin admin] The token cache is being invalidate because project f96f099eae0c4fe58ef30986514abbb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:19:45.187850 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8e1ebf48-7dbb-4e21-8419-ab298c7d11d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:45.188817 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8e1ebf48-7dbb-4e21-8419-ab298c7d11d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:45.190855 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 930/1854] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:19:44 2022] DELETE /identity/v3/projects/f96f099eae0c4fe58ef30986514abbb8 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:45.194493 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd3201e-b0de-4289-91a4-375e4b90e9c5 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:45.196834 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-efd3201e-b0de-4289-91a4-375e4b90e9c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:45.211953 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd3201e-b0de-4289-91a4-375e4b90e9c5 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:45.289965 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c93c44-00f4-4074-a024-262571322320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:45.291037 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c93c44-00f4-4074-a024-262571322320 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:45.291441 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c93c44-00f4-4074-a024-262571322320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:45.291789 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c93c44-00f4-4074-a024-262571322320 admin admin] PATH_INFO: `/v3/users/7af5a088140e4809bb7cc9aa8060fd6d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:45.293278 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c93c44-00f4-4074-a024-262571322320 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7af5a088140e4809bb7cc9aa8060fd6d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:45.295310 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c93c44-00f4-4074-a024-262571322320 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:45.357308 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-32c93c44-00f4-4074-a024-262571322320 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7af5a088140e4809bb7cc9aa8060fd6d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:45.366543 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-32c93c44-00f4-4074-a024-262571322320 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7af5a088140e4809bb7cc9aa8060fd6d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:45.367471 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-32c93c44-00f4-4074-a024-262571322320 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7af5a088140e4809bb7cc9aa8060fd6d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:45.376543 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-32c93c44-00f4-4074-a024-262571322320 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7af5a088140e4809bb7cc9aa8060fd6d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:45.402792 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-32c93c44-00f4-4074-a024-262571322320 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7af5a088140e4809bb7cc9aa8060fd6d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:45.412560 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 925/1855] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:19:45 2022] DELETE /identity/v3/users/7af5a088140e4809bb7cc9aa8060fd6d => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:45.461143 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1ebf48-7dbb-4e21-8419-ab298c7d11d9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:45.462438 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8e1ebf48-7dbb-4e21-8419-ab298c7d11d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:45.486268 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1ebf48-7dbb-4e21-8419-ab298c7d11d9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:45.534810 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c93c44-00f4-4074-a024-262571322320 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:45.536711 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-32c93c44-00f4-4074-a024-262571322320 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:45.595021 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b79619c-cc9d-4124-8e5d-a100f940bdb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:45.595899 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b79619c-cc9d-4124-8e5d-a100f940bdb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:45.596234 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b79619c-cc9d-4124-8e5d-a100f940bdb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:45.596462 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b79619c-cc9d-4124-8e5d-a100f940bdb8 admin admin] PATH_INFO: `/v3/projects/2ae26f7176544e4a9e1d96369edaa354` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:45.597979 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b79619c-cc9d-4124-8e5d-a100f940bdb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ae26f7176544e4a9e1d96369edaa354)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:45.599716 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b79619c-cc9d-4124-8e5d-a100f940bdb8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:45.635375 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c93c44-00f4-4074-a024-262571322320 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:45.642288 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2df395-2f12-425e-b22c-7c32ebbdfea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:45.643174 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2df395-2f12-425e-b22c-7c32ebbdfea4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:45.643464 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2df395-2f12-425e-b22c-7c32ebbdfea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:45.643743 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2df395-2f12-425e-b22c-7c32ebbdfea4 admin admin] PATH_INFO: `/v3/projects/5a4193e976ff42a1bfe19deaca893953` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:45.646436 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2df395-2f12-425e-b22c-7c32ebbdfea4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a4193e976ff42a1bfe19deaca893953)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:45.650653 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2df395-2f12-425e-b22c-7c32ebbdfea4 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:45.663367 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:45.679874 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6b79619c-cc9d-4124-8e5d-a100f940bdb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ae26f7176544e4a9e1d96369edaa354', 'request_id': 'req-6b79619c-cc9d-4124-8e5d-a100f940bdb8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:45.689122 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6b79619c-cc9d-4124-8e5d-a100f940bdb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ae26f7176544e4a9e1d96369edaa354', 'request_id': 'req-6b79619c-cc9d-4124-8e5d-a100f940bdb8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:45.700019 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6b79619c-cc9d-4124-8e5d-a100f940bdb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ae26f7176544e4a9e1d96369edaa354', 'request_id': 'req-6b79619c-cc9d-4124-8e5d-a100f940bdb8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:45.715938 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ec2df395-2f12-425e-b22c-7c32ebbdfea4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a4193e976ff42a1bfe19deaca893953', 'request_id': 'req-ec2df395-2f12-425e-b22c-7c32ebbdfea4'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:45.719516 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6b79619c-cc9d-4124-8e5d-a100f940bdb8 admin admin] The token cache is being invalidate because project 2ae26f7176544e4a9e1d96369edaa354 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:19:45.719941 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6b79619c-cc9d-4124-8e5d-a100f940bdb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:45.720656 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6b79619c-cc9d-4124-8e5d-a100f940bdb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:45.723082 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 931/1856] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:19:45 2022] DELETE /identity/v3/projects/2ae26f7176544e4a9e1d96369edaa354 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:45.729644 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54483072-b8fa-4225-b84e-c5bbd47344eb None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:45.731529 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ec2df395-2f12-425e-b22c-7c32ebbdfea4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a4193e976ff42a1bfe19deaca893953', 'request_id': 'req-ec2df395-2f12-425e-b22c-7c32ebbdfea4'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:45.732541 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54483072-b8fa-4225-b84e-c5bbd47344eb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:45.733251 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54483072-b8fa-4225-b84e-c5bbd47344eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:45.752410 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ec2df395-2f12-425e-b22c-7c32ebbdfea4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a4193e976ff42a1bfe19deaca893953', 'request_id': 'req-ec2df395-2f12-425e-b22c-7c32ebbdfea4'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:45.767560 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-54483072-b8fa-4225-b84e-c5bbd47344eb None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:45.768821 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-54483072-b8fa-4225-b84e-c5bbd47344eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:45.771087 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ec2df395-2f12-425e-b22c-7c32ebbdfea4 admin admin] The token cache is being invalidate because project 5a4193e976ff42a1bfe19deaca893953 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:19:45.772178 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ec2df395-2f12-425e-b22c-7c32ebbdfea4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:45.774283 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ec2df395-2f12-425e-b22c-7c32ebbdfea4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:45.776301 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 926/1857] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:19:45 2022] DELETE /identity/v3/projects/5a4193e976ff42a1bfe19deaca893953 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:19:45.781165 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5807e7cc-af89-4364-adf0-e20bbcf79503 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:45.781478 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5807e7cc-af89-4364-adf0-e20bbcf79503 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:45.781768 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5807e7cc-af89-4364-adf0-e20bbcf79503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:45.825318 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:45.829943 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-5807e7cc-af89-4364-adf0-e20bbcf79503 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:45.832038 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-5807e7cc-af89-4364-adf0-e20bbcf79503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:45.856234 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 932/1858] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:45.864095 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3292a0fe-aedf-42e8-a26e-a31e8d9af1ff None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:45.868643 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3292a0fe-aedf-42e8-a26e-a31e8d9af1ff None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:45.868643 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3292a0fe-aedf-42e8-a26e-a31e8d9af1ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:45.919236 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3292a0fe-aedf-42e8-a26e-a31e8d9af1ff None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:45.922458 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3292a0fe-aedf-42e8-a26e-a31e8d9af1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:45.944645 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 933/1859] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:45.947578 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3292a0fe-aedf-42e8-a26e-a31e8d9af1ff None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:45.950115 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3292a0fe-aedf-42e8-a26e-a31e8d9af1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:45.953603 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 927/1860] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:19:45.954567 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5807e7cc-af89-4364-adf0-e20bbcf79503 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:45.966791 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5807e7cc-af89-4364-adf0-e20bbcf79503 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:45.971212 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3292a0fe-aedf-42e8-a26e-a31e8d9af1ff None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:45.972798 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c5b8f5-0d9f-4ba8-86f4-30f0d08d8173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:45.973731 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5b8f5-0d9f-4ba8-86f4-30f0d08d8173 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:45.974019 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5b8f5-0d9f-4ba8-86f4-30f0d08d8173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:45.974290 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5b8f5-0d9f-4ba8-86f4-30f0d08d8173 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:45.975227 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c5b8f5-0d9f-4ba8-86f4-30f0d08d8173 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:19:45.975300 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c5b8f5-0d9f-4ba8-86f4-30f0d08d8173 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:45.978154 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c5b8f5-0d9f-4ba8-86f4-30f0d08d8173 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:45.995415 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b70b50-30d1-4e8f-a749-2f4a9a8f8055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:45.996058 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 928/1861] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:19:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:19:45.996898 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b70b50-30d1-4e8f-a749-2f4a9a8f8055 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:45.997434 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b70b50-30d1-4e8f-a749-2f4a9a8f8055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:45.997857 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b70b50-30d1-4e8f-a749-2f4a9a8f8055 admin admin] PATH_INFO: `/v3/projects/1ac1170c339f4d5fa4a9ce7cbbdc21e4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:45.999622 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b70b50-30d1-4e8f-a749-2f4a9a8f8055 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ac1170c339f4d5fa4a9ce7cbbdc21e4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:46.001407 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a7a3c-2102-499f-af96-94c036d9b5b3 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:46.001407 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a7a3c-2102-499f-af96-94c036d9b5b3 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:46.003099 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a7a3c-2102-499f-af96-94c036d9b5b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:46.003319 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b70b50-30d1-4e8f-a749-2f4a9a8f8055 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:46.044933 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-877a7a3c-2102-499f-af96-94c036d9b5b3 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:46.046604 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-877a7a3c-2102-499f-af96-94c036d9b5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:46.071131 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-94b70b50-30d1-4e8f-a749-2f4a9a8f8055 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ac1170c339f4d5fa4a9ce7cbbdc21e4', 'request_id': 'req-94b70b50-30d1-4e8f-a749-2f4a9a8f8055'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:46.079545 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-94b70b50-30d1-4e8f-a749-2f4a9a8f8055 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ac1170c339f4d5fa4a9ce7cbbdc21e4', 'request_id': 'req-94b70b50-30d1-4e8f-a749-2f4a9a8f8055'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:46.089693 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-94b70b50-30d1-4e8f-a749-2f4a9a8f8055 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ac1170c339f4d5fa4a9ce7cbbdc21e4', 'request_id': 'req-94b70b50-30d1-4e8f-a749-2f4a9a8f8055'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:46.109909 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-94b70b50-30d1-4e8f-a749-2f4a9a8f8055 admin admin] The token cache is being invalidate because project 1ac1170c339f4d5fa4a9ce7cbbdc21e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:19:46.110206 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-94b70b50-30d1-4e8f-a749-2f4a9a8f8055 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:46.111134 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-94b70b50-30d1-4e8f-a749-2f4a9a8f8055 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:46.117875 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 934/1862] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:19:45 2022] DELETE /identity/v3/projects/1ac1170c339f4d5fa4a9ce7cbbdc21e4 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:46.117875 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b70b50-30d1-4e8f-a749-2f4a9a8f8055 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:46.120200 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-94b70b50-30d1-4e8f-a749-2f4a9a8f8055 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:46.200278 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 929/1863] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:46.200278 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6edefc1-6b33-4f0f-bf44-80214c4da578 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:46.200278 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6edefc1-6b33-4f0f-bf44-80214c4da578 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:46.200278 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6edefc1-6b33-4f0f-bf44-80214c4da578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:46.219285 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-e6edefc1-6b33-4f0f-bf44-80214c4da578 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:46.220630 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e6edefc1-6b33-4f0f-bf44-80214c4da578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:46.233205 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 930/1864] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:46.233205 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6edefc1-6b33-4f0f-bf44-80214c4da578 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:46.244129 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6edefc1-6b33-4f0f-bf44-80214c4da578 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:46.250194 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382155a4-18cd-4e63-a0cc-3470af1346c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:46.251069 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382155a4-18cd-4e63-a0cc-3470af1346c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:46.251502 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b70b50-30d1-4e8f-a749-2f4a9a8f8055 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:46.251687 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382155a4-18cd-4e63-a0cc-3470af1346c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:46.251990 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382155a4-18cd-4e63-a0cc-3470af1346c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:46.252991 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382155a4-18cd-4e63-a0cc-3470af1346c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:19:46.253327 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382155a4-18cd-4e63-a0cc-3470af1346c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:46.255112 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382155a4-18cd-4e63-a0cc-3470af1346c3 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:46.259550 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2449315-aa28-4d80-97c2-a198746d9935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:46.260090 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2449315-aa28-4d80-97c2-a198746d9935 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:46.260337 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2449315-aa28-4d80-97c2-a198746d9935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:46.260617 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2449315-aa28-4d80-97c2-a198746d9935 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:46.261368 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2449315-aa28-4d80-97c2-a198746d9935 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:46.263469 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2449315-aa28-4d80-97c2-a198746d9935 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:46.269912 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 931/1865] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:19:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:19:46.271289 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a2dfe-d58a-4e5a-88e5-e3eaced1f64f None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:46.271585 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a2dfe-d58a-4e5a-88e5-e3eaced1f64f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:46.271891 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a2dfe-d58a-4e5a-88e5-e3eaced1f64f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:46.286241 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:46.294160 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-971a2dfe-d58a-4e5a-88e5-e3eaced1f64f None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:46.296496 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-971a2dfe-d58a-4e5a-88e5-e3eaced1f64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:46.307633 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 932/1866] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:46.308640 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971a2dfe-d58a-4e5a-88e5-e3eaced1f64f None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:46.313080 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 935/1867] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:19:46 2022] POST /identity/v3/projects => generated 387 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:19:46.313080 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2449315-aa28-4d80-97c2-a198746d9935 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:46.325705 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971a2dfe-d58a-4e5a-88e5-e3eaced1f64f None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:46.365925 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:46.381589 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2449315-aa28-4d80-97c2-a198746d9935 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:46.388146 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ab638-ef97-4272-a977-dee321da8cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:46.388984 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ab638-ef97-4272-a977-dee321da8cdb admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:46.389327 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ab638-ef97-4272-a977-dee321da8cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:46.389440 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771fd7f3-5400-4f6b-8184-5d31b7cc5a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:46.389629 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ab638-ef97-4272-a977-dee321da8cdb admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:46.390051 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771fd7f3-5400-4f6b-8184-5d31b7cc5a23 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:46.390337 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771fd7f3-5400-4f6b-8184-5d31b7cc5a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:46.390442 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81ab638-ef97-4272-a977-dee321da8cdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:46.390678 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771fd7f3-5400-4f6b-8184-5d31b7cc5a23 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:46.391420 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771fd7f3-5400-4f6b-8184-5d31b7cc5a23 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:19:46.391870 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771fd7f3-5400-4f6b-8184-5d31b7cc5a23 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:46.392636 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81ab638-ef97-4272-a977-dee321da8cdb admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:46.393390 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771fd7f3-5400-4f6b-8184-5d31b7cc5a23 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:46.409518 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 936/1868] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:19:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:19:46.409518 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771fd7f3-5400-4f6b-8184-5d31b7cc5a23 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:46.418226 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771fd7f3-5400-4f6b-8184-5d31b7cc5a23 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:46.425073 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 933/1869] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:19:46 2022] POST /identity/v3/projects => generated 379 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:19:46.425073 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ab638-ef97-4272-a977-dee321da8cdb admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:46.427049 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c47952-7ae8-4ab9-96f5-bf5f1c3101e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:46.427734 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c47952-7ae8-4ab9-96f5-bf5f1c3101e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:46.428021 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c47952-7ae8-4ab9-96f5-bf5f1c3101e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:46.428307 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c47952-7ae8-4ab9-96f5-bf5f1c3101e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:46.429779 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c47952-7ae8-4ab9-96f5-bf5f1c3101e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:46.431854 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c47952-7ae8-4ab9-96f5-bf5f1c3101e3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:46.468730 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 937/1870] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:19:46 2022] POST /identity/v3/users => generated 294 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:19:46.472739 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c47952-7ae8-4ab9-96f5-bf5f1c3101e3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:46.488180 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c47952-7ae8-4ab9-96f5-bf5f1c3101e3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:46.514579 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ab638-ef97-4272-a977-dee321da8cdb admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:46.521525 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca22e03a-baeb-4e8c-aa67-9d5d9e8c5e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:46.522208 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca22e03a-baeb-4e8c-aa67-9d5d9e8c5e8b admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:46.522493 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca22e03a-baeb-4e8c-aa67-9d5d9e8c5e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:46.523340 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca22e03a-baeb-4e8c-aa67-9d5d9e8c5e8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:46.524107 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca22e03a-baeb-4e8c-aa67-9d5d9e8c5e8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:46.526582 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca22e03a-baeb-4e8c-aa67-9d5d9e8c5e8b admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:46.568357 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661027ce-b0ff-4e63-8519-e82c8231d2ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:46.569224 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661027ce-b0ff-4e63-8519-e82c8231d2ad admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:46.570339 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661027ce-b0ff-4e63-8519-e82c8231d2ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:46.570843 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661027ce-b0ff-4e63-8519-e82c8231d2ad admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:46.572123 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661027ce-b0ff-4e63-8519-e82c8231d2ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:46.574480 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661027ce-b0ff-4e63-8519-e82c8231d2ad admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:46.589394 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 934/1871] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:19:46 2022] POST /identity/v3/projects => generated 371 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:19:46.593734 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca22e03a-baeb-4e8c-aa67-9d5d9e8c5e8b admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:46.608894 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 938/1872] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:19:46 2022] POST /identity/v3/users => generated 290 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:19:46.610343 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661027ce-b0ff-4e63-8519-e82c8231d2ad admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:46.624085 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661027ce-b0ff-4e63-8519-e82c8231d2ad admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:46.700548 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca22e03a-baeb-4e8c-aa67-9d5d9e8c5e8b admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:46.705791 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6cabf0-7b19-4ac0-bf60-946dc550b44a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:46.706654 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9672fc-a5c2-47e2-b449-d4099f230133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:46.707146 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6cabf0-7b19-4ac0-bf60-946dc550b44a admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:46.707326 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9672fc-a5c2-47e2-b449-d4099f230133 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:46.707612 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9672fc-a5c2-47e2-b449-d4099f230133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:46.707889 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9672fc-a5c2-47e2-b449-d4099f230133 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:46.708245 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6cabf0-7b19-4ac0-bf60-946dc550b44a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:46.708628 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9672fc-a5c2-47e2-b449-d4099f230133 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:46.708954 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6cabf0-7b19-4ac0-bf60-946dc550b44a admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:46.709888 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6cabf0-7b19-4ac0-bf60-946dc550b44a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:46.711117 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9672fc-a5c2-47e2-b449-d4099f230133 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:46.713987 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6cabf0-7b19-4ac0-bf60-946dc550b44a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:46.733139 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 935/1873] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:46 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:19:46.733139 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9672fc-a5c2-47e2-b449-d4099f230133 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:46.743945 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9672fc-a5c2-47e2-b449-d4099f230133 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:46.748783 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 939/1874] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:19:46 2022] POST /identity/v3/users => generated 286 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:19:46.750013 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6cabf0-7b19-4ac0-bf60-946dc550b44a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:46.759194 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60a68e1-6cf8-4733-9cfc-dcf312ae81e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:46.761761 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a68e1-6cf8-4733-9cfc-dcf312ae81e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:46.762054 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a68e1-6cf8-4733-9cfc-dcf312ae81e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:46.762334 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a68e1-6cf8-4733-9cfc-dcf312ae81e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:46.763135 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60a68e1-6cf8-4733-9cfc-dcf312ae81e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:46.764989 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60a68e1-6cf8-4733-9cfc-dcf312ae81e9 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:46.766457 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:46.774964 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6cabf0-7b19-4ac0-bf60-946dc550b44a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:46.780112 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 936/1875] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:46 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:19:46.780651 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60a68e1-6cf8-4733-9cfc-dcf312ae81e9 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:46.784625 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48148d67-45bf-4a3d-a892-1961c7a57dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:46.785912 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48148d67-45bf-4a3d-a892-1961c7a57dfb admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:46.786628 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48148d67-45bf-4a3d-a892-1961c7a57dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:46.787200 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48148d67-45bf-4a3d-a892-1961c7a57dfb admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:46.789485 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48148d67-45bf-4a3d-a892-1961c7a57dfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:46.792277 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48148d67-45bf-4a3d-a892-1961c7a57dfb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:46.792491 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60a68e1-6cf8-4733-9cfc-dcf312ae81e9 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:46.797655 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa8cc9c-9b7c-48b2-ba14-76dfcb08c430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:46.798489 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa8cc9c-9b7c-48b2-ba14-76dfcb08c430 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:46.798885 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa8cc9c-9b7c-48b2-ba14-76dfcb08c430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:46.799430 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa8cc9c-9b7c-48b2-ba14-76dfcb08c430 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:46.800443 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa8cc9c-9b7c-48b2-ba14-76dfcb08c430 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:46.802318 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa8cc9c-9b7c-48b2-ba14-76dfcb08c430 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:46.812003 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 940/1876] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:46 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:19:46.814165 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48148d67-45bf-4a3d-a892-1961c7a57dfb admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:46.824213 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 937/1877] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:46 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:19:46.825258 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa8cc9c-9b7c-48b2-ba14-76dfcb08c430 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:46.830970 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48148d67-45bf-4a3d-a892-1961c7a57dfb admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:46.835970 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa8cc9c-9b7c-48b2-ba14-76dfcb08c430 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:46.836925 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8365a9-39c5-4b53-8b90-3970e519aa60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:46.837660 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8365a9-39c5-4b53-8b90-3970e519aa60 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:46.837949 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8365a9-39c5-4b53-8b90-3970e519aa60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:46.838360 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8365a9-39c5-4b53-8b90-3970e519aa60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:46.839353 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8365a9-39c5-4b53-8b90-3970e519aa60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:46.841008 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8365a9-39c5-4b53-8b90-3970e519aa60 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:46.841847 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c6318b-c268-4b41-a741-6d4a3da1cd5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:46.842885 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c6318b-c268-4b41-a741-6d4a3da1cd5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:46.843262 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c6318b-c268-4b41-a741-6d4a3da1cd5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:46.843638 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c6318b-c268-4b41-a741-6d4a3da1cd5b admin admin] PATH_INFO: `/v3/projects/a74cfa6f9a8147fa86bebbb55d8ffe59/users/16f74ee38e1b47b6bdc43b6e143e7399/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:46.854220 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 941/1878] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:46 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:19:46.854711 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8365a9-39c5-4b53-8b90-3970e519aa60 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:46.864367 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c6318b-c268-4b41-a741-6d4a3da1cd5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a74cfa6f9a8147fa86bebbb55d8ffe59, user_id=16f74ee38e1b47b6bdc43b6e143e7399, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:46.865721 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8365a9-39c5-4b53-8b90-3970e519aa60 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:46.868259 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c6318b-c268-4b41-a741-6d4a3da1cd5b admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:46.872615 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964f513a-971f-4381-b960-2bd414cc0bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:46.873518 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f513a-971f-4381-b960-2bd414cc0bb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:46.873945 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f513a-971f-4381-b960-2bd414cc0bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:46.877489 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f513a-971f-4381-b960-2bd414cc0bb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:46.878274 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964f513a-971f-4381-b960-2bd414cc0bb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:46.880301 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964f513a-971f-4381-b960-2bd414cc0bb5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:46.887787 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 938/1879] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:19:46 2022] PUT /identity/v3/projects/a74cfa6f9a8147fa86bebbb55d8ffe59/users/16f74ee38e1b47b6bdc43b6e143e7399/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:46.889455 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c6318b-c268-4b41-a741-6d4a3da1cd5b admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:46.897200 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 942/1880] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:46 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:19:46.898918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964f513a-971f-4381-b960-2bd414cc0bb5 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:46.914229 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964f513a-971f-4381-b960-2bd414cc0bb5 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:46.924971 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:46.983088 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c6318b-c268-4b41-a741-6d4a3da1cd5b admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:46.989796 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66df0ca-c4b1-4312-8d9c-e5b9ba70d771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:46.990722 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddadb4ab-29fd-4f70-b498-0cb1d420ead3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:46.991218 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66df0ca-c4b1-4312-8d9c-e5b9ba70d771 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:46.992201 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66df0ca-c4b1-4312-8d9c-e5b9ba70d771 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:46.998324 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddadb4ab-29fd-4f70-b498-0cb1d420ead3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:46.998324 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddadb4ab-29fd-4f70-b498-0cb1d420ead3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:46.998324 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddadb4ab-29fd-4f70-b498-0cb1d420ead3 admin admin] PATH_INFO: `/v3/projects/f571f9a1b0d04239be18c627f5a985e2/users/f795bb8c30bc4a20a741ea914db1320a/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:47.000005 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66df0ca-c4b1-4312-8d9c-e5b9ba70d771 admin admin] PATH_INFO: `/v3/projects/f8639cdf215b47e4a68b846417671171/users/07ce630a3a054a39b721a22b467ef358/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:47.033341 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddadb4ab-29fd-4f70-b498-0cb1d420ead3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f571f9a1b0d04239be18c627f5a985e2, user_id=f795bb8c30bc4a20a741ea914db1320a, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:47.043122 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddadb4ab-29fd-4f70-b498-0cb1d420ead3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:47.063451 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66df0ca-c4b1-4312-8d9c-e5b9ba70d771 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8639cdf215b47e4a68b846417671171, user_id=07ce630a3a054a39b721a22b467ef358, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:47.072169 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66df0ca-c4b1-4312-8d9c-e5b9ba70d771 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:47.099077 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 943/1881] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:19:46 2022] PUT /identity/v3/projects/f571f9a1b0d04239be18c627f5a985e2/users/f795bb8c30bc4a20a741ea914db1320a/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:47.116384 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0fbd8b-1b85-4b36-ab1a-0c00be4c3e1d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:47.116384 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0fbd8b-1b85-4b36-ab1a-0c00be4c3e1d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:47.116384 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0fbd8b-1b85-4b36-ab1a-0c00be4c3e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:47.177617 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 939/1882] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:19:46 2022] PUT /identity/v3/projects/f8639cdf215b47e4a68b846417671171/users/07ce630a3a054a39b721a22b467ef358/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:47.177617 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c94c444-fd8e-423a-8fc8-3a52566a8a64 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:47.177617 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c94c444-fd8e-423a-8fc8-3a52566a8a64 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:47.177617 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c94c444-fd8e-423a-8fc8-3a52566a8a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:47.194322 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6c0fbd8b-1b85-4b36-ab1a-0c00be4c3e1d None None] MFA Rules not processed for user `16f74ee38e1b47b6bdc43b6e143e7399`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:47.199769 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6c0fbd8b-1b85-4b36-ab1a-0c00be4c3e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:47.207162 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-0c94c444-fd8e-423a-8fc8-3a52566a8a64 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:47.208822 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0c94c444-fd8e-423a-8fc8-3a52566a8a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:47.308304 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 940/1883] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:47.310009 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c37ee0-efed-4807-ae70-2acc199b3037 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:47.310408 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c37ee0-efed-4807-ae70-2acc199b3037 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:47.310852 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c37ee0-efed-4807-ae70-2acc199b3037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:47.368715 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 944/1884] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:47.368715 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0fbd8b-1b85-4b36-ab1a-0c00be4c3e1d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:47.371149 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-15c37ee0-efed-4807-ae70-2acc199b3037 None None] MFA Rules not processed for user `07ce630a3a054a39b721a22b467ef358`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:47.373096 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-15c37ee0-efed-4807-ae70-2acc199b3037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:47.375333 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6c0fbd8b-1b85-4b36-ab1a-0c00be4c3e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:47.470552 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:47.513885 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0fbd8b-1b85-4b36-ab1a-0c00be4c3e1d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:47.525769 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ecdff7-3b1e-49cc-acc0-3267c538b6fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:47.527133 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ecdff7-3b1e-49cc-acc0-3267c538b6fa service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:47.527317 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ecdff7-3b1e-49cc-acc0-3267c538b6fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:47.527557 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ecdff7-3b1e-49cc-acc0-3267c538b6fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:47.550114 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 941/1885] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:47.551754 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c37ee0-efed-4807-ae70-2acc199b3037 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:47.562598 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ecdff7-3b1e-49cc-acc0-3267c538b6fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:47.563989 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ecdff7-3b1e-49cc-acc0-3267c538b6fa service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:47.574172 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c37ee0-efed-4807-ae70-2acc199b3037 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:47.589395 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67214c05-6387-42f4-9d7f-78e1e01df1e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:47.590284 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 945/1886] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:19:47.590852 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67214c05-6387-42f4-9d7f-78e1e01df1e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:47.591043 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ecdff7-3b1e-49cc-acc0-3267c538b6fa service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:47.591147 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67214c05-6387-42f4-9d7f-78e1e01df1e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:47.591533 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67214c05-6387-42f4-9d7f-78e1e01df1e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:47.592242 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67214c05-6387-42f4-9d7f-78e1e01df1e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:47.594519 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67214c05-6387-42f4-9d7f-78e1e01df1e6 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:47.604079 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ecdff7-3b1e-49cc-acc0-3267c538b6fa service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:47.609143 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740ebef1-b149-48be-9d4d-9a4ae397f8ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:47.609816 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ebef1-b149-48be-9d4d-9a4ae397f8ca admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:47.610097 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ebef1-b149-48be-9d4d-9a4ae397f8ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:47.610374 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ebef1-b149-48be-9d4d-9a4ae397f8ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:47.611125 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740ebef1-b149-48be-9d4d-9a4ae397f8ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:47.613574 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740ebef1-b149-48be-9d4d-9a4ae397f8ca admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:47.640127 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 942/1887] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:19:47 2022] POST /identity/v3/projects => generated 385 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:19:47.641884 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67214c05-6387-42f4-9d7f-78e1e01df1e6 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:47.644910 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-67214c05-6387-42f4-9d7f-78e1e01df1e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:47.654497 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 946/1888] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:19:47 2022] POST /identity/v3/projects => generated 379 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:19:47.654497 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740ebef1-b149-48be-9d4d-9a4ae397f8ca admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:47.660257 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:47.726661 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740ebef1-b149-48be-9d4d-9a4ae397f8ca admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:47.731793 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4387da5-9750-4855-bff1-560bf05e311c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:47.732485 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4387da5-9750-4855-bff1-560bf05e311c admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:47.732827 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4387da5-9750-4855-bff1-560bf05e311c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:47.733193 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4387da5-9750-4855-bff1-560bf05e311c admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:47.733892 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4387da5-9750-4855-bff1-560bf05e311c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:47.735864 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4387da5-9750-4855-bff1-560bf05e311c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:47.744358 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67214c05-6387-42f4-9d7f-78e1e01df1e6 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:47.751178 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2116ca6f-d11e-4f2c-b090-bd34aa33fba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:47.752372 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2116ca6f-d11e-4f2c-b090-bd34aa33fba2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:47.752712 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2116ca6f-d11e-4f2c-b090-bd34aa33fba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:47.753003 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2116ca6f-d11e-4f2c-b090-bd34aa33fba2 admin admin] PATH_INFO: `/v3/users/a98f8c510ce3485a99298d4d904f8b1d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:47.754431 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2116ca6f-d11e-4f2c-b090-bd34aa33fba2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a98f8c510ce3485a99298d4d904f8b1d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:47.756624 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2116ca6f-d11e-4f2c-b090-bd34aa33fba2 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:47.759471 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 947/1889] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:19:47 2022] POST /identity/v3/users => generated 299 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:19:47.760625 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4387da5-9750-4855-bff1-560bf05e311c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:47.783707 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4387da5-9750-4855-bff1-560bf05e311c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:47.790866 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606f1fd4-5b06-49c4-95e3-5d3e3fdad5e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:47.791710 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f1fd4-5b06-49c4-95e3-5d3e3fdad5e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:47.792063 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f1fd4-5b06-49c4-95e3-5d3e3fdad5e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:47.792460 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f1fd4-5b06-49c4-95e3-5d3e3fdad5e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:47.793557 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606f1fd4-5b06-49c4-95e3-5d3e3fdad5e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:47.797144 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606f1fd4-5b06-49c4-95e3-5d3e3fdad5e8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:47.826907 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-2116ca6f-d11e-4f2c-b090-bd34aa33fba2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a98f8c510ce3485a99298d4d904f8b1d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:47.843396 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 948/1890] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:19:47 2022] POST /identity/v3/users => generated 296 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:19:47.847126 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606f1fd4-5b06-49c4-95e3-5d3e3fdad5e8 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:47.847727 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-2116ca6f-d11e-4f2c-b090-bd34aa33fba2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a98f8c510ce3485a99298d4d904f8b1d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:47.849801 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-2116ca6f-d11e-4f2c-b090-bd34aa33fba2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a98f8c510ce3485a99298d4d904f8b1d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:47.863566 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606f1fd4-5b06-49c4-95e3-5d3e3fdad5e8 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:47.871047 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc7f6dc-c8d6-4e30-b25a-38f75004a2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:47.871746 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7f6dc-c8d6-4e30-b25a-38f75004a2f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:47.872032 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7f6dc-c8d6-4e30-b25a-38f75004a2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:47.872311 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7f6dc-c8d6-4e30-b25a-38f75004a2f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:47.873739 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc7f6dc-c8d6-4e30-b25a-38f75004a2f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:47.873853 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-2116ca6f-d11e-4f2c-b090-bd34aa33fba2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a98f8c510ce3485a99298d4d904f8b1d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:47.875358 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc7f6dc-c8d6-4e30-b25a-38f75004a2f3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:47.892165 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 949/1891] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:47 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:19:47.893329 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc7f6dc-c8d6-4e30-b25a-38f75004a2f3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:47.895464 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2dc7f6dc-c8d6-4e30-b25a-38f75004a2f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:47.901524 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-2116ca6f-d11e-4f2c-b090-bd34aa33fba2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a98f8c510ce3485a99298d4d904f8b1d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:47.910439 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 943/1892] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:19:47 2022] DELETE /identity/v3/users/a98f8c510ce3485a99298d4d904f8b1d => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:47.912849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc7f6dc-c8d6-4e30-b25a-38f75004a2f3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:47.916775 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2116ca6f-d11e-4f2c-b090-bd34aa33fba2 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:47.918434 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-2116ca6f-d11e-4f2c-b090-bd34aa33fba2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:48.001840 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:48.012948 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe7d0c4-bdab-4b52-acb7-dd1f965b484f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:48.013718 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7d0c4-bdab-4b52-acb7-dd1f965b484f admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:48.014147 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7d0c4-bdab-4b52-acb7-dd1f965b484f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:48.014564 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7d0c4-bdab-4b52-acb7-dd1f965b484f admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:48.015308 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe7d0c4-bdab-4b52-acb7-dd1f965b484f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:48.016985 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe7d0c4-bdab-4b52-acb7-dd1f965b484f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:48.030562 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 950/1893] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:47 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:19:48.032118 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6016b20b-ccb1-4589-a09a-28e90c018787 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:48.032419 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6016b20b-ccb1-4589-a09a-28e90c018787 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:48.032704 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6016b20b-ccb1-4589-a09a-28e90c018787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:48.070385 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6016b20b-ccb1-4589-a09a-28e90c018787 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:48.072072 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6016b20b-ccb1-4589-a09a-28e90c018787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:48.077460 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2116ca6f-d11e-4f2c-b090-bd34aa33fba2 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:48.087196 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983c71e2-bbc0-4698-bc95-bbb1bfa61f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:48.088130 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c71e2-bbc0-4698-bc95-bbb1bfa61f10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:48.088417 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c71e2-bbc0-4698-bc95-bbb1bfa61f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:48.089850 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c71e2-bbc0-4698-bc95-bbb1bfa61f10 admin admin] PATH_INFO: `/v3/projects/c713365bf746462490511dba3370bd04/users/7bff5ea9007f44fdb5930754fca34719/roles/f36e053b442540f08041ac7878760f65` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:48.095124 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 951/1894] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:48.095124 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6016b20b-ccb1-4589-a09a-28e90c018787 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:48.118950 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6016b20b-ccb1-4589-a09a-28e90c018787 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:48.128586 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983c71e2-bbc0-4698-bc95-bbb1bfa61f10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c713365bf746462490511dba3370bd04, user_id=7bff5ea9007f44fdb5930754fca34719, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:48.132024 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983c71e2-bbc0-4698-bc95-bbb1bfa61f10 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:48.136258 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2741fa79-a28c-4c31-a0b8-138bf0c0c42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:48.137212 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741fa79-a28c-4c31-a0b8-138bf0c0c42c admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:48.137498 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741fa79-a28c-4c31-a0b8-138bf0c0c42c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:48.137778 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741fa79-a28c-4c31-a0b8-138bf0c0c42c admin admin] PATH_INFO: `/v3/projects/c316d5b9bcb64d70bb51bce12cb54cac/users/3edcfaa8139842488bb6ca61ba5a37d7/roles/f36e053b442540f08041ac7878760f65` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:48.164153 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2741fa79-a28c-4c31-a0b8-138bf0c0c42c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c316d5b9bcb64d70bb51bce12cb54cac, user_id=3edcfaa8139842488bb6ca61ba5a37d7, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:48.173789 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 944/1895] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:19:47 2022] PUT /identity/v3/projects/c713365bf746462490511dba3370bd04/users/7bff5ea9007f44fdb5930754fca34719/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:48.174836 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983c71e2-bbc0-4698-bc95-bbb1bfa61f10 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:48.178463 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-983c71e2-bbc0-4698-bc95-bbb1bfa61f10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:48.181776 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2741fa79-a28c-4c31-a0b8-138bf0c0c42c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:48.221208 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 952/1896] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:19:48 2022] PUT /identity/v3/projects/c316d5b9bcb64d70bb51bce12cb54cac/users/3edcfaa8139842488bb6ca61ba5a37d7/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:48.222990 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2741fa79-a28c-4c31-a0b8-138bf0c0c42c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:48.282527 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:48.297251 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983c71e2-bbc0-4698-bc95-bbb1bfa61f10 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:48.325981 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2741fa79-a28c-4c31-a0b8-138bf0c0c42c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:48.330887 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2072de-5140-44f1-9f5f-cafe8a264799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:48.331498 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2072de-5140-44f1-9f5f-cafe8a264799 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:48.331789 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2072de-5140-44f1-9f5f-cafe8a264799 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:48.332101 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2072de-5140-44f1-9f5f-cafe8a264799 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:48.332800 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2072de-5140-44f1-9f5f-cafe8a264799 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:48.334510 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2072de-5140-44f1-9f5f-cafe8a264799 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:48.355889 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 953/1897] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:48 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:19:48.357120 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2072de-5140-44f1-9f5f-cafe8a264799 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:48.367816 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504ad794-1935-4a18-8480-f5c1310e31a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:48.368706 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ad794-1935-4a18-8480-f5c1310e31a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:48.369336 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ad794-1935-4a18-8480-f5c1310e31a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:48.369455 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ad794-1935-4a18-8480-f5c1310e31a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:48.371660 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2072de-5140-44f1-9f5f-cafe8a264799 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:48.376872 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ca3fd1-67cb-4933-977b-9fbe40b9dcc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:48.378165 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ca3fd1-67cb-4933-977b-9fbe40b9dcc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:48.378457 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ca3fd1-67cb-4933-977b-9fbe40b9dcc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:48.378899 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ca3fd1-67cb-4933-977b-9fbe40b9dcc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:48.379675 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504ad794-1935-4a18-8480-f5c1310e31a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:48.379794 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ca3fd1-67cb-4933-977b-9fbe40b9dcc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:48.381154 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504ad794-1935-4a18-8480-f5c1310e31a0 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:48.387601 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ca3fd1-67cb-4933-977b-9fbe40b9dcc1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:48.402322 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 945/1898] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:19:48.403150 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504ad794-1935-4a18-8480-f5c1310e31a0 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:48.413138 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 954/1899] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:48 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:19:48.414797 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504ad794-1935-4a18-8480-f5c1310e31a0 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:48.416438 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ca3fd1-67cb-4933-977b-9fbe40b9dcc1 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:48.420364 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec452a5-25e7-4a3d-8951-6800c0c54fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:48.421488 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec452a5-25e7-4a3d-8951-6800c0c54fa6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:48.421871 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec452a5-25e7-4a3d-8951-6800c0c54fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:48.422366 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec452a5-25e7-4a3d-8951-6800c0c54fa6 admin admin] PATH_INFO: `/v3/projects/c713365bf746462490511dba3370bd04/users/7bff5ea9007f44fdb5930754fca34719/roles/f36e053b442540f08041ac7878760f65` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:48.424503 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec452a5-25e7-4a3d-8951-6800c0c54fa6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c713365bf746462490511dba3370bd04, user_id=7bff5ea9007f44fdb5930754fca34719, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:48.427426 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec452a5-25e7-4a3d-8951-6800c0c54fa6 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:48.431931 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ca3fd1-67cb-4933-977b-9fbe40b9dcc1 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:48.440047 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 946/1900] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:19:48 2022] PUT /identity/v3/projects/c713365bf746462490511dba3370bd04/users/7bff5ea9007f44fdb5930754fca34719/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:48.451300 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3316ba8e-dbe7-42d8-b377-e44d9e917882 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:48.451600 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3316ba8e-dbe7-42d8-b377-e44d9e917882 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:48.452022 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3316ba8e-dbe7-42d8-b377-e44d9e917882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:48.485852 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-3316ba8e-dbe7-42d8-b377-e44d9e917882 None None] MFA Rules not processed for user `7bff5ea9007f44fdb5930754fca34719`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:48.487074 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-3316ba8e-dbe7-42d8-b377-e44d9e917882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:48.500178 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6056c5f-ed71-4e14-ae85-313c6c4fd1b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:48.501208 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6056c5f-ed71-4e14-ae85-313c6c4fd1b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:48.501589 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6056c5f-ed71-4e14-ae85-313c6c4fd1b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:48.501944 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6056c5f-ed71-4e14-ae85-313c6c4fd1b8 admin admin] PATH_INFO: `/v3/projects/c316d5b9bcb64d70bb51bce12cb54cac/users/3edcfaa8139842488bb6ca61ba5a37d7/roles/f36e053b442540f08041ac7878760f65` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:48.504167 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6056c5f-ed71-4e14-ae85-313c6c4fd1b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c316d5b9bcb64d70bb51bce12cb54cac, user_id=3edcfaa8139842488bb6ca61ba5a37d7, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:48.507643 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6056c5f-ed71-4e14-ae85-313c6c4fd1b8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:48.535567 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 955/1901] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:19:48 2022] PUT /identity/v3/projects/c316d5b9bcb64d70bb51bce12cb54cac/users/3edcfaa8139842488bb6ca61ba5a37d7/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:48.538288 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e3a18-2aff-4c97-9590-2a32a03df82b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:48.538424 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e3a18-2aff-4c97-9590-2a32a03df82b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:48.539310 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e3a18-2aff-4c97-9590-2a32a03df82b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:48.549202 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:48.593077 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 947/1902] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:48 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:48.603917 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3316ba8e-dbe7-42d8-b377-e44d9e917882 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:48.604864 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3f4e3a18-2aff-4c97-9590-2a32a03df82b None None] MFA Rules not processed for user `3edcfaa8139842488bb6ca61ba5a37d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:48.605551 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-3316ba8e-dbe7-42d8-b377-e44d9e917882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:48.606165 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3f4e3a18-2aff-4c97-9590-2a32a03df82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:48.702630 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3316ba8e-dbe7-42d8-b377-e44d9e917882 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:48.706887 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 956/1903] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:48 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:48.721163 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4e3a18-2aff-4c97-9590-2a32a03df82b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:48.723081 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3f4e3a18-2aff-4c97-9590-2a32a03df82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:48.778682 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6180df6b-1cc3-4b0c-a162-61c4a7ffb8fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:48.779955 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6180df6b-1cc3-4b0c-a162-61c4a7ffb8fd service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:48.780473 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6180df6b-1cc3-4b0c-a162-61c4a7ffb8fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:48.780828 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6180df6b-1cc3-4b0c-a162-61c4a7ffb8fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:48.786644 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:48.813399 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4e3a18-2aff-4c97-9590-2a32a03df82b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:48.817982 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff4a18e-67c2-49c5-a7ca-71ce49aa1a49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:48.818658 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff4a18e-67c2-49c5-a7ca-71ce49aa1a49 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:48.819078 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff4a18e-67c2-49c5-a7ca-71ce49aa1a49 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:48.819367 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff4a18e-67c2-49c5-a7ca-71ce49aa1a49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:48.834597 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff4a18e-67c2-49c5-a7ca-71ce49aa1a49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:48.837249 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff4a18e-67c2-49c5-a7ca-71ce49aa1a49 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:48.858522 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 957/1904] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:48 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:19:48.871375 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6180df6b-1cc3-4b0c-a162-61c4a7ffb8fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:48.873146 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6180df6b-1cc3-4b0c-a162-61c4a7ffb8fd service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:48.887998 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 948/1905] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:48 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:19:49.008563 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff4a18e-67c2-49c5-a7ca-71ce49aa1a49 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:49.126463 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6180df6b-1cc3-4b0c-a162-61c4a7ffb8fd service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:49.129911 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-6180df6b-1cc3-4b0c-a162-61c4a7ffb8fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:49.199991 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff4a18e-67c2-49c5-a7ca-71ce49aa1a49 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:49.214424 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3716946-33b5-4024-98ee-74c32205846d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:49.216296 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3716946-33b5-4024-98ee-74c32205846d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:49.216584 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3716946-33b5-4024-98ee-74c32205846d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:49.216873 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3716946-33b5-4024-98ee-74c32205846d admin admin] PATH_INFO: `/v3/users/16f74ee38e1b47b6bdc43b6e143e7399` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:49.220731 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3716946-33b5-4024-98ee-74c32205846d admin admin] RBAC: Authorizing `identity:delete_user(user_id=16f74ee38e1b47b6bdc43b6e143e7399)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:49.229581 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3716946-33b5-4024-98ee-74c32205846d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:49.283910 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:49.330022 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6180df6b-1cc3-4b0c-a162-61c4a7ffb8fd service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:49.335576 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f72fae0-db66-4131-9bdc-fe50774db511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:49.336549 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72fae0-db66-4131-9bdc-fe50774db511 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:49.336833 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72fae0-db66-4131-9bdc-fe50774db511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:49.338131 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72fae0-db66-4131-9bdc-fe50774db511 admin admin] PATH_INFO: `/v3/projects/ba522c5410c84b8583d9d766309483cc` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:49.340418 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f72fae0-db66-4131-9bdc-fe50774db511 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba522c5410c84b8583d9d766309483cc)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:49.343429 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b3716946-33b5-4024-98ee-74c32205846d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16f74ee38e1b47b6bdc43b6e143e7399'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:49.343565 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f72fae0-db66-4131-9bdc-fe50774db511 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:49.344414 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b3716946-33b5-4024-98ee-74c32205846d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16f74ee38e1b47b6bdc43b6e143e7399'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:49.373080 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b3716946-33b5-4024-98ee-74c32205846d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16f74ee38e1b47b6bdc43b6e143e7399'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:49.406772 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b3716946-33b5-4024-98ee-74c32205846d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16f74ee38e1b47b6bdc43b6e143e7399'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:49.407802 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b3716946-33b5-4024-98ee-74c32205846d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16f74ee38e1b47b6bdc43b6e143e7399'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:49.429888 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 958/1906] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:19:49 2022] DELETE /identity/v3/users/16f74ee38e1b47b6bdc43b6e143e7399 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:49.434392 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87e0e02-a059-4b90-be41-1b0882f5c6f3 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:49.434825 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87e0e02-a059-4b90-be41-1b0882f5c6f3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:49.436176 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87e0e02-a059-4b90-be41-1b0882f5c6f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:49.462213 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0f72fae0-db66-4131-9bdc-fe50774db511 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba522c5410c84b8583d9d766309483cc', 'request_id': 'req-0f72fae0-db66-4131-9bdc-fe50774db511'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:49.465095 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e87e0e02-a059-4b90-be41-1b0882f5c6f3 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:49.466776 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e87e0e02-a059-4b90-be41-1b0882f5c6f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:49.484098 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0f72fae0-db66-4131-9bdc-fe50774db511 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba522c5410c84b8583d9d766309483cc', 'request_id': 'req-0f72fae0-db66-4131-9bdc-fe50774db511'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:49.496425 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0f72fae0-db66-4131-9bdc-fe50774db511 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba522c5410c84b8583d9d766309483cc', 'request_id': 'req-0f72fae0-db66-4131-9bdc-fe50774db511'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:49.550458 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0f72fae0-db66-4131-9bdc-fe50774db511 admin admin] The token cache is being invalidate because project ba522c5410c84b8583d9d766309483cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:19:49.551350 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0f72fae0-db66-4131-9bdc-fe50774db511 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:49.552718 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0f72fae0-db66-4131-9bdc-fe50774db511 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:49.555021 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 949/1907] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:19:49 2022] DELETE /identity/v3/projects/ba522c5410c84b8583d9d766309483cc => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:49.556144 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f72fae0-db66-4131-9bdc-fe50774db511 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:49.557996 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0f72fae0-db66-4131-9bdc-fe50774db511 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:49.566493 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:49.592907 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 959/1908] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:49.594196 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87e0e02-a059-4b90-be41-1b0882f5c6f3 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:49.595837 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e87e0e02-a059-4b90-be41-1b0882f5c6f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:49.622043 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87e0e02-a059-4b90-be41-1b0882f5c6f3 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:49.627077 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0b64ca-82f7-40d9-9ff2-0ac73a00e2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:49.628200 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b64ca-82f7-40d9-9ff2-0ac73a00e2cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:49.628487 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b64ca-82f7-40d9-9ff2-0ac73a00e2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:49.628771 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b64ca-82f7-40d9-9ff2-0ac73a00e2cb admin admin] PATH_INFO: `/v3/users/0cb6c64c64fa4caa94e9ec91d0eeb693` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:49.630106 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0b64ca-82f7-40d9-9ff2-0ac73a00e2cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cb6c64c64fa4caa94e9ec91d0eeb693)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:49.632594 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0b64ca-82f7-40d9-9ff2-0ac73a00e2cb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:49.673896 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f72fae0-db66-4131-9bdc-fe50774db511 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:49.680293 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08071dc-0435-4ced-97c9-6701166f4daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:49.681625 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08071dc-0435-4ced-97c9-6701166f4daf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:49.681907 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08071dc-0435-4ced-97c9-6701166f4daf admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:49.682190 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08071dc-0435-4ced-97c9-6701166f4daf admin admin] PATH_INFO: `/v3/users/07ce630a3a054a39b721a22b467ef358` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:49.683536 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08071dc-0435-4ced-97c9-6701166f4daf admin admin] RBAC: Authorizing `identity:delete_user(user_id=07ce630a3a054a39b721a22b467ef358)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:49.687127 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08071dc-0435-4ced-97c9-6701166f4daf admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:49.702191 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4b0b64ca-82f7-40d9-9ff2-0ac73a00e2cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cb6c64c64fa4caa94e9ec91d0eeb693'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:49.705536 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4b0b64ca-82f7-40d9-9ff2-0ac73a00e2cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cb6c64c64fa4caa94e9ec91d0eeb693'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:49.717202 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4b0b64ca-82f7-40d9-9ff2-0ac73a00e2cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cb6c64c64fa4caa94e9ec91d0eeb693'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:49.735736 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4b0b64ca-82f7-40d9-9ff2-0ac73a00e2cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cb6c64c64fa4caa94e9ec91d0eeb693'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:49.737509 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4b0b64ca-82f7-40d9-9ff2-0ac73a00e2cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cb6c64c64fa4caa94e9ec91d0eeb693'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:49.755904 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 960/1909] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:19:49 2022] DELETE /identity/v3/users/0cb6c64c64fa4caa94e9ec91d0eeb693 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:49.757489 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0b64ca-82f7-40d9-9ff2-0ac73a00e2cb admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:49.760524 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c08071dc-0435-4ced-97c9-6701166f4daf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07ce630a3a054a39b721a22b467ef358'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:49.772037 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4b0b64ca-82f7-40d9-9ff2-0ac73a00e2cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:49.773808 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c08071dc-0435-4ced-97c9-6701166f4daf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07ce630a3a054a39b721a22b467ef358'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:49.775874 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c08071dc-0435-4ced-97c9-6701166f4daf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07ce630a3a054a39b721a22b467ef358'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:49.798643 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c08071dc-0435-4ced-97c9-6701166f4daf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07ce630a3a054a39b721a22b467ef358'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:49.836591 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c08071dc-0435-4ced-97c9-6701166f4daf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07ce630a3a054a39b721a22b467ef358'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:49.842254 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:49.858814 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 950/1910] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:19:49 2022] DELETE /identity/v3/users/07ce630a3a054a39b721a22b467ef358 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:49.861910 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57544ca-b9ca-4c02-9f33-e88369ae1377 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:49.862204 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57544ca-b9ca-4c02-9f33-e88369ae1377 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:49.867140 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57544ca-b9ca-4c02-9f33-e88369ae1377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:49.922623 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-e57544ca-b9ca-4c02-9f33-e88369ae1377 None None] MFA Rules not processed for user `f795bb8c30bc4a20a741ea914db1320a`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:49.923862 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e57544ca-b9ca-4c02-9f33-e88369ae1377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:49.927611 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0b64ca-82f7-40d9-9ff2-0ac73a00e2cb admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:49.992020 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc14fd8-d1b0-4a04-96bf-d0d1620170d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:49.993025 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc14fd8-d1b0-4a04-96bf-d0d1620170d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:49.993161 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc14fd8-d1b0-4a04-96bf-d0d1620170d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:49.993859 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc14fd8-d1b0-4a04-96bf-d0d1620170d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:49.996606 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ecc14fd8-d1b0-4a04-96bf-d0d1620170d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:50.070622 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 951/1911] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:50.072033 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a93112-1042-4a7e-acdd-177b8fdd477c None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:50.072373 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a93112-1042-4a7e-acdd-177b8fdd477c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:50.072591 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a93112-1042-4a7e-acdd-177b8fdd477c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:50.074746 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:50.118757 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-a2a93112-1042-4a7e-acdd-177b8fdd477c None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:50.120031 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a2a93112-1042-4a7e-acdd-177b8fdd477c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:50.131083 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc14fd8-d1b0-4a04-96bf-d0d1620170d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:50.132637 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc14fd8-d1b0-4a04-96bf-d0d1620170d4 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:50.140296 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 952/1912] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:50.140296 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57163808-efde-4ba6-9feb-3094cabb3279 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:50.140296 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57163808-efde-4ba6-9feb-3094cabb3279 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:50.140296 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57163808-efde-4ba6-9feb-3094cabb3279 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:50.149559 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 953/1913] 10.222.0.71 () {54 vars in 921 bytes} [Mon Jul 4 09:19:50 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:19:50.150399 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57163808-efde-4ba6-9feb-3094cabb3279 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:50.153174 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-57163808-efde-4ba6-9feb-3094cabb3279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:50.160633 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 961/1914] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:19:50.162176 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc14fd8-d1b0-4a04-96bf-d0d1620170d4 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:50.166032 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ecc14fd8-d1b0-4a04-96bf-d0d1620170d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:50.180733 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc14fd8-d1b0-4a04-96bf-d0d1620170d4 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:50.191442 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bffc45d-1ea5-4b1c-ace6-793fcf1000a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:50.192126 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bffc45d-1ea5-4b1c-ace6-793fcf1000a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:50.192415 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bffc45d-1ea5-4b1c-ace6-793fcf1000a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:50.192700 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bffc45d-1ea5-4b1c-ace6-793fcf1000a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:50.205086 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bffc45d-1ea5-4b1c-ace6-793fcf1000a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:50.206865 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bffc45d-1ea5-4b1c-ace6-793fcf1000a2 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:50.207183 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57163808-efde-4ba6-9feb-3094cabb3279 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:50.219384 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437c2f56-956f-4a8e-868a-2c6f04f5cbbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:50.220137 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c2f56-956f-4a8e-868a-2c6f04f5cbbb admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:50.220478 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c2f56-956f-4a8e-868a-2c6f04f5cbbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:50.220766 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c2f56-956f-4a8e-868a-2c6f04f5cbbb admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:50.221690 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437c2f56-956f-4a8e-868a-2c6f04f5cbbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:50.221846 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 962/1915] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:19:50.223228 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad2978-3cbe-4f04-8ed1-4c23214878fc None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:50.223567 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad2978-3cbe-4f04-8ed1-4c23214878fc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:50.223904 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad2978-3cbe-4f04-8ed1-4c23214878fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:50.227069 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437c2f56-956f-4a8e-868a-2c6f04f5cbbb admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:50.252746 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a1ad2978-3cbe-4f04-8ed1-4c23214878fc None None] MFA Rules not processed for user `15039da209ec445d93d49e2bd6d854eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:50.254128 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a1ad2978-3cbe-4f04-8ed1-4c23214878fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:50.289649 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 954/1916] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:19:50 2022] POST /identity/v3/projects => generated 371 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:19:50.291206 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437c2f56-956f-4a8e-868a-2c6f04f5cbbb admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:50.399241 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 963/1917] 10.222.0.71 () {58 vars in 1018 bytes} [Mon Jul 4 09:19:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:50.415263 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ad2978-3cbe-4f04-8ed1-4c23214878fc None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:50.420339 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a1ad2978-3cbe-4f04-8ed1-4c23214878fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:50.426514 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437c2f56-956f-4a8e-868a-2c6f04f5cbbb admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:50.432755 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3bbc09-da4a-4524-bace-09de1b905847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:50.433653 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3bbc09-da4a-4524-bace-09de1b905847 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:50.434045 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3bbc09-da4a-4524-bace-09de1b905847 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:50.434425 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3bbc09-da4a-4524-bace-09de1b905847 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:50.435273 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3bbc09-da4a-4524-bace-09de1b905847 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:50.437510 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3bbc09-da4a-4524-bace-09de1b905847 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:50.454811 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ad2978-3cbe-4f04-8ed1-4c23214878fc None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:50.473017 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 955/1918] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:19:50 2022] POST /identity/v3/users => generated 292 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:19:50.474379 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3bbc09-da4a-4524-bace-09de1b905847 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:50.494311 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3bbc09-da4a-4524-bace-09de1b905847 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:50.507601 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0032a7-9f86-48f8-b010-edd95ff8f2c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:50.508499 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0032a7-9f86-48f8-b010-edd95ff8f2c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:50.508784 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0032a7-9f86-48f8-b010-edd95ff8f2c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:50.509304 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0032a7-9f86-48f8-b010-edd95ff8f2c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:50.510904 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0032a7-9f86-48f8-b010-edd95ff8f2c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:50.515507 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0032a7-9f86-48f8-b010-edd95ff8f2c6 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:50.536068 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 956/1919] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:50 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:19:50.541284 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0032a7-9f86-48f8-b010-edd95ff8f2c6 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:50.559448 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0032a7-9f86-48f8-b010-edd95ff8f2c6 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:50.578845 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e391bf-8d5f-411e-8fe6-dab3e6d6096b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:50.580289 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e391bf-8d5f-411e-8fe6-dab3e6d6096b admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:50.580289 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e391bf-8d5f-411e-8fe6-dab3e6d6096b admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:50.580619 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e391bf-8d5f-411e-8fe6-dab3e6d6096b admin admin] PATH_INFO: `/v3/projects/6b42826f396a4391bef96910cfb8e1fb/users/f645f4a8ac2243da8bcc989cdc59c309/roles/f36e053b442540f08041ac7878760f65` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:50.610559 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dbd025-d5cd-4099-881b-ceecad433f34 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15039da209ec445d93d49e2bd6d854eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:50.611423 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dbd025-d5cd-4099-881b-ceecad433f34 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:50.611815 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dbd025-d5cd-4099-881b-ceecad433f34 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:50.612293 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dbd025-d5cd-4099-881b-ceecad433f34 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:50.614258 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e391bf-8d5f-411e-8fe6-dab3e6d6096b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b42826f396a4391bef96910cfb8e1fb, user_id=f645f4a8ac2243da8bcc989cdc59c309, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:50.623187 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e391bf-8d5f-411e-8fe6-dab3e6d6096b admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:50.633150 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dbd025-d5cd-4099-881b-ceecad433f34 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:50.635893 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dbd025-d5cd-4099-881b-ceecad433f34 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:50.655444 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 957/1920] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:19:50 2022] PUT /identity/v3/projects/6b42826f396a4391bef96910cfb8e1fb/users/f645f4a8ac2243da8bcc989cdc59c309/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:50.655444 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e391bf-8d5f-411e-8fe6-dab3e6d6096b admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:50.663006 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-05e391bf-8d5f-411e-8fe6-dab3e6d6096b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:50.675927 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 964/1921] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:19:50.677240 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dbd025-d5cd-4099-881b-ceecad433f34 service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:50.704208 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:50.705154 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dbd025-d5cd-4099-881b-ceecad433f34 service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:50.797561 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:50.821697 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e391bf-8d5f-411e-8fe6-dab3e6d6096b admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:50.826253 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef5cd4-2fba-435b-aa5e-580016de0aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:50.827107 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef5cd4-2fba-435b-aa5e-580016de0aa9 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:50.827541 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef5cd4-2fba-435b-aa5e-580016de0aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:50.827925 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef5cd4-2fba-435b-aa5e-580016de0aa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:50.828666 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bef5cd4-2fba-435b-aa5e-580016de0aa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:50.830764 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bef5cd4-2fba-435b-aa5e-580016de0aa9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:50.843590 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279391c6-bc56-4b03-a7ea-bd2e23dee3c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:50.844605 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 965/1922] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:50 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:19:50.846279 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279391c6-bc56-4b03-a7ea-bd2e23dee3c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:50.846678 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279391c6-bc56-4b03-a7ea-bd2e23dee3c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:50.847108 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279391c6-bc56-4b03-a7ea-bd2e23dee3c8 admin admin] PATH_INFO: `/v3/projects/3b3aa61f57ee4e4b9cbb6da78de2c577` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:50.848862 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef5cd4-2fba-435b-aa5e-580016de0aa9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:50.850404 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279391c6-bc56-4b03-a7ea-bd2e23dee3c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b3aa61f57ee4e4b9cbb6da78de2c577)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:50.857125 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279391c6-bc56-4b03-a7ea-bd2e23dee3c8 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:50.872751 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef5cd4-2fba-435b-aa5e-580016de0aa9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:50.887776 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcbd22a-b91b-4986-9109-a9067990cd21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:50.889910 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcbd22a-b91b-4986-9109-a9067990cd21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:50.890258 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcbd22a-b91b-4986-9109-a9067990cd21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:50.890889 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcbd22a-b91b-4986-9109-a9067990cd21 admin admin] PATH_INFO: `/v3/projects/6b42826f396a4391bef96910cfb8e1fb/users/f645f4a8ac2243da8bcc989cdc59c309/roles/f36e053b442540f08041ac7878760f65` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:50.925932 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcbd22a-b91b-4986-9109-a9067990cd21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b42826f396a4391bef96910cfb8e1fb, user_id=f645f4a8ac2243da8bcc989cdc59c309, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:50.937136 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcbd22a-b91b-4986-9109-a9067990cd21 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:50.946463 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-279391c6-bc56-4b03-a7ea-bd2e23dee3c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b3aa61f57ee4e4b9cbb6da78de2c577', 'request_id': 'req-279391c6-bc56-4b03-a7ea-bd2e23dee3c8'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:50.956035 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-279391c6-bc56-4b03-a7ea-bd2e23dee3c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b3aa61f57ee4e4b9cbb6da78de2c577', 'request_id': 'req-279391c6-bc56-4b03-a7ea-bd2e23dee3c8'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:50.966206 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 966/1923] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:19:50 2022] PUT /identity/v3/projects/6b42826f396a4391bef96910cfb8e1fb/users/f645f4a8ac2243da8bcc989cdc59c309/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:50.970408 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-279391c6-bc56-4b03-a7ea-bd2e23dee3c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b3aa61f57ee4e4b9cbb6da78de2c577', 'request_id': 'req-279391c6-bc56-4b03-a7ea-bd2e23dee3c8'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:50.995300 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-279391c6-bc56-4b03-a7ea-bd2e23dee3c8 admin admin] The token cache is being invalidate because project 3b3aa61f57ee4e4b9cbb6da78de2c577 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:19:50.996011 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-279391c6-bc56-4b03-a7ea-bd2e23dee3c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:50.996898 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-279391c6-bc56-4b03-a7ea-bd2e23dee3c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:50.999213 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 958/1924] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:19:50 2022] DELETE /identity/v3/projects/3b3aa61f57ee4e4b9cbb6da78de2c577 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:51.003981 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b837d5-7a4b-4b93-9538-fca73858cc1b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:51.004350 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b837d5-7a4b-4b93-9538-fca73858cc1b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:51.004735 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b837d5-7a4b-4b93-9538-fca73858cc1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:51.037630 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f9b837d5-7a4b-4b93-9538-fca73858cc1b None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:51.039418 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f9b837d5-7a4b-4b93-9538-fca73858cc1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:51.167440 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 967/1925] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:51.170238 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fba44c-e4da-4aca-b7e3-3e0f736b0a5f None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:51.170840 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fba44c-e4da-4aca-b7e3-3e0f736b0a5f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:51.171432 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fba44c-e4da-4aca-b7e3-3e0f736b0a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:51.198442 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-70fba44c-e4da-4aca-b7e3-3e0f736b0a5f None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:51.200778 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-70fba44c-e4da-4aca-b7e3-3e0f736b0a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:51.214789 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b837d5-7a4b-4b93-9538-fca73858cc1b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:51.216734 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f9b837d5-7a4b-4b93-9538-fca73858cc1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:51.223292 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 959/1926] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:51.224625 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fba44c-e4da-4aca-b7e3-3e0f736b0a5f None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:51.236936 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fba44c-e4da-4aca-b7e3-3e0f736b0a5f None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:51.246577 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caffc300-de6d-42f1-9489-14b30c11b739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:51.247305 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caffc300-de6d-42f1-9489-14b30c11b739 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:51.247595 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caffc300-de6d-42f1-9489-14b30c11b739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:51.247871 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caffc300-de6d-42f1-9489-14b30c11b739 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:51.248612 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caffc300-de6d-42f1-9489-14b30c11b739 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:19:51.249472 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b837d5-7a4b-4b93-9538-fca73858cc1b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:51.256469 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caffc300-de6d-42f1-9489-14b30c11b739 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:51.260675 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caffc300-de6d-42f1-9489-14b30c11b739 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:51.264552 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dfd756-da79-4fa5-82b3-b8e50e711d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:51.266504 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dfd756-da79-4fa5-82b3-b8e50e711d0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:51.266793 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dfd756-da79-4fa5-82b3-b8e50e711d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:51.268613 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dfd756-da79-4fa5-82b3-b8e50e711d0a admin admin] PATH_INFO: `/v3/projects/a74cfa6f9a8147fa86bebbb55d8ffe59` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:51.270813 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dfd756-da79-4fa5-82b3-b8e50e711d0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a74cfa6f9a8147fa86bebbb55d8ffe59)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:51.277566 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dfd756-da79-4fa5-82b3-b8e50e711d0a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:51.295670 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 960/1927] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:19:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:19:51.300368 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caffc300-de6d-42f1-9489-14b30c11b739 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:51.303531 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-caffc300-de6d-42f1-9489-14b30c11b739 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:51.325989 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caffc300-de6d-42f1-9489-14b30c11b739 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:51.360976 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2ec50d-26e9-49b2-b573-fd485eb12541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:51.364956 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2ec50d-26e9-49b2-b573-fd485eb12541 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:51.365499 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2ec50d-26e9-49b2-b573-fd485eb12541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:51.366050 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2ec50d-26e9-49b2-b573-fd485eb12541 admin admin] PATH_INFO: `/v3/projects/f8639cdf215b47e4a68b846417671171` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:51.370867 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2ec50d-26e9-49b2-b573-fd485eb12541 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8639cdf215b47e4a68b846417671171)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:51.376497 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2ec50d-26e9-49b2-b573-fd485eb12541 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:51.432914 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b5dfd756-da79-4fa5-82b3-b8e50e711d0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a74cfa6f9a8147fa86bebbb55d8ffe59', 'request_id': 'req-b5dfd756-da79-4fa5-82b3-b8e50e711d0a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:51.453478 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b5dfd756-da79-4fa5-82b3-b8e50e711d0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a74cfa6f9a8147fa86bebbb55d8ffe59', 'request_id': 'req-b5dfd756-da79-4fa5-82b3-b8e50e711d0a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:51.457204 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-9a2ec50d-26e9-49b2-b573-fd485eb12541 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8639cdf215b47e4a68b846417671171', 'request_id': 'req-9a2ec50d-26e9-49b2-b573-fd485eb12541'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:51.465521 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-9a2ec50d-26e9-49b2-b573-fd485eb12541 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8639cdf215b47e4a68b846417671171', 'request_id': 'req-9a2ec50d-26e9-49b2-b573-fd485eb12541'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:51.466215 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b5dfd756-da79-4fa5-82b3-b8e50e711d0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a74cfa6f9a8147fa86bebbb55d8ffe59', 'request_id': 'req-b5dfd756-da79-4fa5-82b3-b8e50e711d0a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:51.473556 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-9a2ec50d-26e9-49b2-b573-fd485eb12541 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8639cdf215b47e4a68b846417671171', 'request_id': 'req-9a2ec50d-26e9-49b2-b573-fd485eb12541'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:51.494971 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-9a2ec50d-26e9-49b2-b573-fd485eb12541 admin admin] The token cache is being invalidate because project f8639cdf215b47e4a68b846417671171 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:19:51.496108 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-9a2ec50d-26e9-49b2-b573-fd485eb12541 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:51.497293 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-9a2ec50d-26e9-49b2-b573-fd485eb12541 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:51.499087 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 961/1928] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:19:51 2022] DELETE /identity/v3/projects/f8639cdf215b47e4a68b846417671171 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:51.502828 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2ec50d-26e9-49b2-b573-fd485eb12541 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:51.503294 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b5dfd756-da79-4fa5-82b3-b8e50e711d0a admin admin] The token cache is being invalidate because project a74cfa6f9a8147fa86bebbb55d8ffe59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:19:51.504048 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b5dfd756-da79-4fa5-82b3-b8e50e711d0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:51.504802 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-9a2ec50d-26e9-49b2-b573-fd485eb12541 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:51.505807 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b5dfd756-da79-4fa5-82b3-b8e50e711d0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:51.513278 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 968/1929] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:19:51 2022] DELETE /identity/v3/projects/a74cfa6f9a8147fa86bebbb55d8ffe59 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:51.513278 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dfd756-da79-4fa5-82b3-b8e50e711d0a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:51.514969 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b5dfd756-da79-4fa5-82b3-b8e50e711d0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:51.526303 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:51.625821 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2ec50d-26e9-49b2-b573-fd485eb12541 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:51.633369 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc4a960-3f96-427e-9370-23d05b5791f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:51.634408 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc4a960-3f96-427e-9370-23d05b5791f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:51.634816 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc4a960-3f96-427e-9370-23d05b5791f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:51.635109 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc4a960-3f96-427e-9370-23d05b5791f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:51.636006 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc4a960-3f96-427e-9370-23d05b5791f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:51.638238 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc4a960-3f96-427e-9370-23d05b5791f8 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:51.640759 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:51.678491 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dfd756-da79-4fa5-82b3-b8e50e711d0a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:51.695907 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c1e442-abbd-4ac7-918a-49ecaed3b42d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:51.699906 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 962/1930] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:19:51 2022] POST /identity/v3/projects => generated 387 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:19:51.700146 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c1e442-abbd-4ac7-918a-49ecaed3b42d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:51.700146 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c1e442-abbd-4ac7-918a-49ecaed3b42d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:51.700304 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc4a960-3f96-427e-9370-23d05b5791f8 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:51.700375 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c1e442-abbd-4ac7-918a-49ecaed3b42d admin admin] PATH_INFO: `/v3/users/3edcfaa8139842488bb6ca61ba5a37d7` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:51.702607 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c1e442-abbd-4ac7-918a-49ecaed3b42d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3edcfaa8139842488bb6ca61ba5a37d7)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:51.704556 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c1e442-abbd-4ac7-918a-49ecaed3b42d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:51.705891 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-6cc4a960-3f96-427e-9370-23d05b5791f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:51.789828 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c2c1e442-abbd-4ac7-918a-49ecaed3b42d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3edcfaa8139842488bb6ca61ba5a37d7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:51.793079 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c2c1e442-abbd-4ac7-918a-49ecaed3b42d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3edcfaa8139842488bb6ca61ba5a37d7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:51.803200 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c2c1e442-abbd-4ac7-918a-49ecaed3b42d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3edcfaa8139842488bb6ca61ba5a37d7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:51.823324 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c2c1e442-abbd-4ac7-918a-49ecaed3b42d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3edcfaa8139842488bb6ca61ba5a37d7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:51.825409 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c2c1e442-abbd-4ac7-918a-49ecaed3b42d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3edcfaa8139842488bb6ca61ba5a37d7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:51.833523 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc4a960-3f96-427e-9370-23d05b5791f8 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:51.849081 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba48239e-2a87-4e2d-950e-7021c9a1f740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:51.850107 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48239e-2a87-4e2d-950e-7021c9a1f740 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:51.851862 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48239e-2a87-4e2d-950e-7021c9a1f740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:51.852261 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48239e-2a87-4e2d-950e-7021c9a1f740 admin admin] PATH_INFO: `/v3/users/7bff5ea9007f44fdb5930754fca34719` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:51.857393 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 969/1931] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:19:51 2022] DELETE /identity/v3/users/3edcfaa8139842488bb6ca61ba5a37d7 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:51.857592 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba48239e-2a87-4e2d-950e-7021c9a1f740 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bff5ea9007f44fdb5930754fca34719)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:51.857706 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c1e442-abbd-4ac7-918a-49ecaed3b42d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:51.857796 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c2c1e442-abbd-4ac7-918a-49ecaed3b42d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:51.861367 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba48239e-2a87-4e2d-950e-7021c9a1f740 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:51.940262 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ba48239e-2a87-4e2d-950e-7021c9a1f740 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bff5ea9007f44fdb5930754fca34719'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:51.949621 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c1e442-abbd-4ac7-918a-49ecaed3b42d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:51.956555 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564302f1-4562-41d9-b717-e03c335d7acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:51.957432 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564302f1-4562-41d9-b717-e03c335d7acb admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:51.957741 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564302f1-4562-41d9-b717-e03c335d7acb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:51.958064 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564302f1-4562-41d9-b717-e03c335d7acb admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:51.958652 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ba48239e-2a87-4e2d-950e-7021c9a1f740 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bff5ea9007f44fdb5930754fca34719'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:51.959144 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564302f1-4562-41d9-b717-e03c335d7acb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:51.959641 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ba48239e-2a87-4e2d-950e-7021c9a1f740 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bff5ea9007f44fdb5930754fca34719'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:51.961529 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564302f1-4562-41d9-b717-e03c335d7acb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:51.968215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ba48239e-2a87-4e2d-950e-7021c9a1f740 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bff5ea9007f44fdb5930754fca34719'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:51.986502 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ba48239e-2a87-4e2d-950e-7021c9a1f740 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bff5ea9007f44fdb5930754fca34719'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:51.988864 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 970/1932] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:19:51 2022] POST /identity/v3/users => generated 294 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:19:51.994378 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564302f1-4562-41d9-b717-e03c335d7acb admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:51.997145 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-564302f1-4562-41d9-b717-e03c335d7acb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:51.997663 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 963/1933] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:19:51 2022] DELETE /identity/v3/users/7bff5ea9007f44fdb5930754fca34719 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:52.015954 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564302f1-4562-41d9-b717-e03c335d7acb admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:52.099885 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:52.112531 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a7755-8d36-4234-b3dc-13a6e58cf7a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:52.113827 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a7755-8d36-4234-b3dc-13a6e58cf7a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:52.114128 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a7755-8d36-4234-b3dc-13a6e58cf7a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:52.114412 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a7755-8d36-4234-b3dc-13a6e58cf7a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:52.115164 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9a7755-8d36-4234-b3dc-13a6e58cf7a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:52.117409 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9a7755-8d36-4234-b3dc-13a6e58cf7a3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:52.135008 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba48239e-2a87-4e2d-950e-7021c9a1f740 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:52.147185 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 971/1934] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:51 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:19:52.149925 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba48239e-2a87-4e2d-950e-7021c9a1f740 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:52.156893 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bcd5f1-5244-42d7-844e-4991026383c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:52.157618 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bcd5f1-5244-42d7-844e-4991026383c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:52.158070 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bcd5f1-5244-42d7-844e-4991026383c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:52.158359 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bcd5f1-5244-42d7-844e-4991026383c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:52.159261 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bcd5f1-5244-42d7-844e-4991026383c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:52.161161 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bcd5f1-5244-42d7-844e-4991026383c4 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:52.184657 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 964/1935] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:52 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:19:52.185800 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a7755-8d36-4234-b3dc-13a6e58cf7a3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:52.199510 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a7755-8d36-4234-b3dc-13a6e58cf7a3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:52.205875 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e40c8d-ffd7-4b8f-96e6-2e8717cd5424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:52.207007 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e40c8d-ffd7-4b8f-96e6-2e8717cd5424 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:52.207396 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e40c8d-ffd7-4b8f-96e6-2e8717cd5424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:52.208070 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e40c8d-ffd7-4b8f-96e6-2e8717cd5424 admin admin] PATH_INFO: `/v3/projects/24b88a9e7afd4ac0bd003bb3f0f283f8/users/80feb91acc33494080a74f1305fe404e/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:52.235759 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e40c8d-ffd7-4b8f-96e6-2e8717cd5424 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24b88a9e7afd4ac0bd003bb3f0f283f8, user_id=80feb91acc33494080a74f1305fe404e, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:52.239972 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e40c8d-ffd7-4b8f-96e6-2e8717cd5424 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:52.259581 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 972/1936] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:19:52 2022] PUT /identity/v3/projects/24b88a9e7afd4ac0bd003bb3f0f283f8/users/80feb91acc33494080a74f1305fe404e/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:52.272951 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb05b532-abb8-4cb5-99cc-0ca069bcaee8 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:52.272951 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb05b532-abb8-4cb5-99cc-0ca069bcaee8 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:52.272951 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb05b532-abb8-4cb5-99cc-0ca069bcaee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:52.280366 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:52.308571 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-bb05b532-abb8-4cb5-99cc-0ca069bcaee8 None None] MFA Rules not processed for user `80feb91acc33494080a74f1305fe404e`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:52.310014 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-bb05b532-abb8-4cb5-99cc-0ca069bcaee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:52.354288 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff57025-0d2e-4ea4-82c0-f78dd3301ea5 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:52.354976 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff57025-0d2e-4ea4-82c0-f78dd3301ea5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:52.355125 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff57025-0d2e-4ea4-82c0-f78dd3301ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:52.405770 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8ff57025-0d2e-4ea4-82c0-f78dd3301ea5 None None] MFA Rules not processed for user `f645f4a8ac2243da8bcc989cdc59c309`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:52.407165 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8ff57025-0d2e-4ea4-82c0-f78dd3301ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:52.436396 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 965/1937] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:52.437697 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb05b532-abb8-4cb5-99cc-0ca069bcaee8 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:52.511621 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 973/1938] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:52 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:19:52.512502 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff57025-0d2e-4ea4-82c0-f78dd3301ea5 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:52.513971 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8ff57025-0d2e-4ea4-82c0-f78dd3301ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:52.518083 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb05b532-abb8-4cb5-99cc-0ca069bcaee8 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:52.525223 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c24150-ea86-4ff3-b08a-31d526d7a8ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:52.526200 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c24150-ea86-4ff3-b08a-31d526d7a8ca admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:52.526819 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c24150-ea86-4ff3-b08a-31d526d7a8ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:52.527326 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c24150-ea86-4ff3-b08a-31d526d7a8ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:52.528224 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c24150-ea86-4ff3-b08a-31d526d7a8ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:52.531306 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c24150-ea86-4ff3-b08a-31d526d7a8ca admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:52.574863 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 966/1939] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:19:52 2022] POST /identity/v3/projects => generated 387 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:19:52.576754 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c24150-ea86-4ff3-b08a-31d526d7a8ca admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:52.610913 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff57025-0d2e-4ea4-82c0-f78dd3301ea5 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:52.693891 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c24150-ea86-4ff3-b08a-31d526d7a8ca admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:52.703476 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2ed6aa-cccf-447f-ab55-c275e05b7262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:52.704641 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ed6aa-cccf-447f-ab55-c275e05b7262 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:52.704641 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ed6aa-cccf-447f-ab55-c275e05b7262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:52.705056 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ed6aa-cccf-447f-ab55-c275e05b7262 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:52.705894 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2ed6aa-cccf-447f-ab55-c275e05b7262 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:52.708860 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2ed6aa-cccf-447f-ab55-c275e05b7262 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:52.716265 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2cd65e-d80e-43d0-b9db-8ebe1c628ca2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:52.716941 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2cd65e-d80e-43d0-b9db-8ebe1c628ca2 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:52.717386 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2cd65e-d80e-43d0-b9db-8ebe1c628ca2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:52.717845 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2cd65e-d80e-43d0-b9db-8ebe1c628ca2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:52.720062 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9e2cd65e-d80e-43d0-b9db-8ebe1c628ca2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:52.752788 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:52.754838 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 967/1940] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:19:52 2022] POST /identity/v3/users => generated 300 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:19:52.756019 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2ed6aa-cccf-447f-ab55-c275e05b7262 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:52.760423 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-fb2ed6aa-cccf-447f-ab55-c275e05b7262 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:52.779271 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2ed6aa-cccf-447f-ab55-c275e05b7262 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:52.785799 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccde748f-b33c-4f61-8389-e9c7526908d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:52.786927 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde748f-b33c-4f61-8389-e9c7526908d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:52.800385 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde748f-b33c-4f61-8389-e9c7526908d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:52.801574 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde748f-b33c-4f61-8389-e9c7526908d9 admin admin] PATH_INFO: `/v3/projects/c316d5b9bcb64d70bb51bce12cb54cac` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:52.804833 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccde748f-b33c-4f61-8389-e9c7526908d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c316d5b9bcb64d70bb51bce12cb54cac)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:52.805549 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2cd65e-d80e-43d0-b9db-8ebe1c628ca2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:52.807014 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2cd65e-d80e-43d0-b9db-8ebe1c628ca2 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:52.810214 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccde748f-b33c-4f61-8389-e9c7526908d9 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:52.815053 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:52.822384 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 974/1941] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:19:52.823273 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2cd65e-d80e-43d0-b9db-8ebe1c628ca2 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:52.824664 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9e2cd65e-d80e-43d0-b9db-8ebe1c628ca2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:52.846975 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2cd65e-d80e-43d0-b9db-8ebe1c628ca2 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:52.855275 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5631d6f5-873a-4196-a7d7-2cb9292f6a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:52.856791 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5631d6f5-873a-4196-a7d7-2cb9292f6a51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:52.857216 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5631d6f5-873a-4196-a7d7-2cb9292f6a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:52.857871 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5631d6f5-873a-4196-a7d7-2cb9292f6a51 admin admin] PATH_INFO: `/v3/projects/c713365bf746462490511dba3370bd04` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:52.859763 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5631d6f5-873a-4196-a7d7-2cb9292f6a51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c713365bf746462490511dba3370bd04)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:52.863271 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5631d6f5-873a-4196-a7d7-2cb9292f6a51 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:52.924420 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ccde748f-b33c-4f61-8389-e9c7526908d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c316d5b9bcb64d70bb51bce12cb54cac', 'request_id': 'req-ccde748f-b33c-4f61-8389-e9c7526908d9'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:52.932554 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ccde748f-b33c-4f61-8389-e9c7526908d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c316d5b9bcb64d70bb51bce12cb54cac', 'request_id': 'req-ccde748f-b33c-4f61-8389-e9c7526908d9'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:52.939365 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5631d6f5-873a-4196-a7d7-2cb9292f6a51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c713365bf746462490511dba3370bd04', 'request_id': 'req-5631d6f5-873a-4196-a7d7-2cb9292f6a51'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:52.941456 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ccde748f-b33c-4f61-8389-e9c7526908d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c316d5b9bcb64d70bb51bce12cb54cac', 'request_id': 'req-ccde748f-b33c-4f61-8389-e9c7526908d9'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:52.957763 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ccde748f-b33c-4f61-8389-e9c7526908d9 admin admin] The token cache is being invalidate because project c316d5b9bcb64d70bb51bce12cb54cac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:19:52.958390 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ccde748f-b33c-4f61-8389-e9c7526908d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:52.959039 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ccde748f-b33c-4f61-8389-e9c7526908d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:52.964974 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 968/1942] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:19:52 2022] DELETE /identity/v3/projects/c316d5b9bcb64d70bb51bce12cb54cac => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:52.964974 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccde748f-b33c-4f61-8389-e9c7526908d9 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:52.964974 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ccde748f-b33c-4f61-8389-e9c7526908d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:52.968184 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5631d6f5-873a-4196-a7d7-2cb9292f6a51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c713365bf746462490511dba3370bd04', 'request_id': 'req-5631d6f5-873a-4196-a7d7-2cb9292f6a51'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:52.978511 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5631d6f5-873a-4196-a7d7-2cb9292f6a51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c713365bf746462490511dba3370bd04', 'request_id': 'req-5631d6f5-873a-4196-a7d7-2cb9292f6a51'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:52.997711 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5631d6f5-873a-4196-a7d7-2cb9292f6a51 admin admin] The token cache is being invalidate because project c713365bf746462490511dba3370bd04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:19:52.998005 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5631d6f5-873a-4196-a7d7-2cb9292f6a51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:52.999307 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5631d6f5-873a-4196-a7d7-2cb9292f6a51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:53.002284 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 975/1943] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:19:52 2022] DELETE /identity/v3/projects/c713365bf746462490511dba3370bd04 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:53.004281 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5bd8d4-e3a9-442b-a303-a24849a7696d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:53.004588 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5bd8d4-e3a9-442b-a303-a24849a7696d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:53.005012 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5bd8d4-e3a9-442b-a303-a24849a7696d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:53.032709 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6d5bd8d4-e3a9-442b-a303-a24849a7696d None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:53.034072 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6d5bd8d4-e3a9-442b-a303-a24849a7696d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:53.062067 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccde748f-b33c-4f61-8389-e9c7526908d9 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:53.128544 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9bcc9d-f4f6-4599-a63c-ea272895fd3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:53.129365 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9bcc9d-f4f6-4599-a63c-ea272895fd3b admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:53.129579 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9bcc9d-f4f6-4599-a63c-ea272895fd3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:53.129863 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9bcc9d-f4f6-4599-a63c-ea272895fd3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:53.130600 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9bcc9d-f4f6-4599-a63c-ea272895fd3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:53.133091 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9bcc9d-f4f6-4599-a63c-ea272895fd3b admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:53.138488 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 976/1944] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:53.140368 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d93ba5-26b1-4d88-8d21-733808ab2078 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:53.140906 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d93ba5-26b1-4d88-8d21-733808ab2078 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:53.141357 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d93ba5-26b1-4d88-8d21-733808ab2078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:53.149222 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 969/1945] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:52 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:19:53.153457 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce2849a-1529-4776-828f-7c6c1f056a28 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:53.153832 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce2849a-1529-4776-828f-7c6c1f056a28 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:53.154200 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce2849a-1529-4776-828f-7c6c1f056a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:53.172557 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-94d93ba5-26b1-4d88-8d21-733808ab2078 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:53.173812 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-94d93ba5-26b1-4d88-8d21-733808ab2078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:53.174955 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-9ce2849a-1529-4776-828f-7c6c1f056a28 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:53.176193 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-9ce2849a-1529-4776-828f-7c6c1f056a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:53.183972 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 977/1946] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:53.186215 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d93ba5-26b1-4d88-8d21-733808ab2078 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:53.186379 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 970/1947] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:53.187934 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-94d93ba5-26b1-4d88-8d21-733808ab2078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:53.191416 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1b6f8-47f1-440a-b4c2-05be41d08d56 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:53.191922 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1b6f8-47f1-440a-b4c2-05be41d08d56 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:53.192368 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1b6f8-47f1-440a-b4c2-05be41d08d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:53.211749 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d93ba5-26b1-4d88-8d21-733808ab2078 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:53.216329 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-d5f1b6f8-47f1-440a-b4c2-05be41d08d56 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:53.218070 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d5f1b6f8-47f1-440a-b4c2-05be41d08d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:53.218721 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed21552-b87d-4374-8955-4370f0ef9f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:53.219985 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed21552-b87d-4374-8955-4370f0ef9f2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:53.220613 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed21552-b87d-4374-8955-4370f0ef9f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:53.221057 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed21552-b87d-4374-8955-4370f0ef9f2f admin admin] PATH_INFO: `/v3/projects/e97cea692c284df1b938ee46bda369b0/users/c3127dea8f084a4c8e59321507192892/roles/f36e053b442540f08041ac7878760f65` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:53.228636 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 971/1948] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:53.229604 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f1b6f8-47f1-440a-b4c2-05be41d08d56 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:53.243300 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f1b6f8-47f1-440a-b4c2-05be41d08d56 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:53.245653 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed21552-b87d-4374-8955-4370f0ef9f2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e97cea692c284df1b938ee46bda369b0, user_id=c3127dea8f084a4c8e59321507192892, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:53.249340 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed21552-b87d-4374-8955-4370f0ef9f2f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:53.261247 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7c419e-5b1b-4d86-a153-b59fbc75bca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:53.261918 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7c419e-5b1b-4d86-a153-b59fbc75bca1 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:53.262212 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7c419e-5b1b-4d86-a153-b59fbc75bca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:53.262514 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7c419e-5b1b-4d86-a153-b59fbc75bca1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:53.263228 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7c419e-5b1b-4d86-a153-b59fbc75bca1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:19:53.263541 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7c419e-5b1b-4d86-a153-b59fbc75bca1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:53.266220 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7c419e-5b1b-4d86-a153-b59fbc75bca1 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:53.267561 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 978/1949] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:19:53 2022] PUT /identity/v3/projects/e97cea692c284df1b938ee46bda369b0/users/c3127dea8f084a4c8e59321507192892/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:53.268590 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed21552-b87d-4374-8955-4370f0ef9f2f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:53.294906 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:53.307094 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 972/1950] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:19:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:19:53.308316 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7c419e-5b1b-4d86-a153-b59fbc75bca1 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:53.322557 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7c419e-5b1b-4d86-a153-b59fbc75bca1 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:53.326929 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:53.352076 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed21552-b87d-4374-8955-4370f0ef9f2f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:53.356849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e2a70-0f46-4cf3-98b9-2c85a902611c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:53.357542 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e2a70-0f46-4cf3-98b9-2c85a902611c admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:53.357828 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e2a70-0f46-4cf3-98b9-2c85a902611c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:53.358104 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e2a70-0f46-4cf3-98b9-2c85a902611c admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:53.358836 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2e2a70-0f46-4cf3-98b9-2c85a902611c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:19:53.359139 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2e2a70-0f46-4cf3-98b9-2c85a902611c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:53.360723 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2e2a70-0f46-4cf3-98b9-2c85a902611c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:53.373849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 979/1951] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:19:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:19:53.374891 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e2a70-0f46-4cf3-98b9-2c85a902611c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:53.387365 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e2a70-0f46-4cf3-98b9-2c85a902611c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:53.388338 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2116a44-5ba5-4e17-a00f-8a1a40a962cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:53.389138 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2116a44-5ba5-4e17-a00f-8a1a40a962cb admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:53.389429 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2116a44-5ba5-4e17-a00f-8a1a40a962cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:53.389710 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2116a44-5ba5-4e17-a00f-8a1a40a962cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:53.390453 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2116a44-5ba5-4e17-a00f-8a1a40a962cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:53.392366 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72bd692-8637-4621-a92e-b5a00c2111bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:53.393019 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72bd692-8637-4621-a92e-b5a00c2111bc admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:53.393330 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72bd692-8637-4621-a92e-b5a00c2111bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:53.393611 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72bd692-8637-4621-a92e-b5a00c2111bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:53.394738 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72bd692-8637-4621-a92e-b5a00c2111bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:53.396212 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72bd692-8637-4621-a92e-b5a00c2111bc admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:53.397852 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2116a44-5ba5-4e17-a00f-8a1a40a962cb admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:53.409463 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 973/1952] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:53 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:19:53.410358 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2116a44-5ba5-4e17-a00f-8a1a40a962cb admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:53.422575 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2116a44-5ba5-4e17-a00f-8a1a40a962cb admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:53.428016 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295d0fa6-26c2-4f42-8d8e-f57cc7f4809a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:53.429427 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295d0fa6-26c2-4f42-8d8e-f57cc7f4809a admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:53.429594 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 980/1953] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:19:53 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:19:53.430207 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295d0fa6-26c2-4f42-8d8e-f57cc7f4809a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:53.430207 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295d0fa6-26c2-4f42-8d8e-f57cc7f4809a admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:53.430388 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72bd692-8637-4621-a92e-b5a00c2111bc admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:53.430661 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295d0fa6-26c2-4f42-8d8e-f57cc7f4809a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:53.432881 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295d0fa6-26c2-4f42-8d8e-f57cc7f4809a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:53.467368 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 974/1954] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:19:53 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:19:53.467368 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295d0fa6-26c2-4f42-8d8e-f57cc7f4809a admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:53.505532 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72bd692-8637-4621-a92e-b5a00c2111bc admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:53.542193 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295d0fa6-26c2-4f42-8d8e-f57cc7f4809a admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:53.548137 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76afd110-c403-4c49-a790-520859952fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:53.548816 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76afd110-c403-4c49-a790-520859952fcf admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:53.549146 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76afd110-c403-4c49-a790-520859952fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:53.549411 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76afd110-c403-4c49-a790-520859952fcf admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:53.550106 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76afd110-c403-4c49-a790-520859952fcf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:53.552097 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76afd110-c403-4c49-a790-520859952fcf admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:53.578927 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 975/1955] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:19:53 2022] POST /identity/v3/users => generated 292 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:19:53.580515 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76afd110-c403-4c49-a790-520859952fcf admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:53.595213 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76afd110-c403-4c49-a790-520859952fcf admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:53.600829 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1396e328-d2a6-4b8a-b040-85ebcf01f6e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:53.602190 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1396e328-d2a6-4b8a-b040-85ebcf01f6e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:53.602482 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1396e328-d2a6-4b8a-b040-85ebcf01f6e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:53.603393 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1396e328-d2a6-4b8a-b040-85ebcf01f6e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:53.603578 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1396e328-d2a6-4b8a-b040-85ebcf01f6e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:53.605649 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1396e328-d2a6-4b8a-b040-85ebcf01f6e3 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:53.608228 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a597bd4e-2bb6-468f-aa63-a2cd53300553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:53.609294 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a597bd4e-2bb6-468f-aa63-a2cd53300553 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:53.609589 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a597bd4e-2bb6-468f-aa63-a2cd53300553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:53.609861 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a597bd4e-2bb6-468f-aa63-a2cd53300553 admin admin] PATH_INFO: `/v3/projects/e97cea692c284df1b938ee46bda369b0/users/c3127dea8f084a4c8e59321507192892/roles/f36e053b442540f08041ac7878760f65` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:53.614406 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a597bd4e-2bb6-468f-aa63-a2cd53300553 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e97cea692c284df1b938ee46bda369b0, user_id=c3127dea8f084a4c8e59321507192892, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:53.622572 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a597bd4e-2bb6-468f-aa63-a2cd53300553 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:53.636897 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 981/1956] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:19:53 2022] PUT /identity/v3/projects/e97cea692c284df1b938ee46bda369b0/users/c3127dea8f084a4c8e59321507192892/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:53.637820 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a597bd4e-2bb6-468f-aa63-a2cd53300553 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:53.652656 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 976/1957] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:19:53 2022] POST /identity/v3/users => generated 291 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:19:53.654737 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f44d78-348a-4725-b9a9-0cee9623a5d8 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:53.655044 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f44d78-348a-4725-b9a9-0cee9623a5d8 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:53.655324 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f44d78-348a-4725-b9a9-0cee9623a5d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:53.696546 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-f5f44d78-348a-4725-b9a9-0cee9623a5d8 None None] MFA Rules not processed for user `c3127dea8f084a4c8e59321507192892`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:53.698054 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f5f44d78-348a-4725-b9a9-0cee9623a5d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:53.706856 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:53.761591 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:53.804253 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 977/1958] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:53 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:53.804253 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f44d78-348a-4725-b9a9-0cee9623a5d8 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:53.814224 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f44d78-348a-4725-b9a9-0cee9623a5d8 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:53.823861 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5995566-8c6d-4fac-ac23-79efa932c445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:53.824579 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5995566-8c6d-4fac-ac23-79efa932c445 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:53.825085 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5995566-8c6d-4fac-ac23-79efa932c445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:53.825351 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5995566-8c6d-4fac-ac23-79efa932c445 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:53.826109 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5995566-8c6d-4fac-ac23-79efa932c445 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:53.827271 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a597bd4e-2bb6-468f-aa63-a2cd53300553 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:53.832254 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5995566-8c6d-4fac-ac23-79efa932c445 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:53.842067 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899bb7a5-e160-4413-b04f-6f7d31540850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:53.842781 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899bb7a5-e160-4413-b04f-6f7d31540850 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:53.843068 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899bb7a5-e160-4413-b04f-6f7d31540850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:53.843350 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899bb7a5-e160-4413-b04f-6f7d31540850 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:53.844099 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899bb7a5-e160-4413-b04f-6f7d31540850 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:53.845940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899bb7a5-e160-4413-b04f-6f7d31540850 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:53.860511 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 982/1959] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:53 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:19:53.869244 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899bb7a5-e160-4413-b04f-6f7d31540850 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:53.869418 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 978/1960] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:53 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:19:53.869418 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5995566-8c6d-4fac-ac23-79efa932c445 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:53.870532 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-899bb7a5-e160-4413-b04f-6f7d31540850 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:53.892525 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5995566-8c6d-4fac-ac23-79efa932c445 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:53.906465 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8cd734-7f34-4de0-bb4e-c5908d3b3b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:53.907917 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8cd734-7f34-4de0-bb4e-c5908d3b3b29 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:53.908424 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8cd734-7f34-4de0-bb4e-c5908d3b3b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:53.909627 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8cd734-7f34-4de0-bb4e-c5908d3b3b29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:53.910652 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8cd734-7f34-4de0-bb4e-c5908d3b3b29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:53.912573 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8cd734-7f34-4de0-bb4e-c5908d3b3b29 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:53.936288 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 979/1961] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:53 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:19:53.939269 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8cd734-7f34-4de0-bb4e-c5908d3b3b29 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:53.974255 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8cd734-7f34-4de0-bb4e-c5908d3b3b29 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:53.992545 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19614250-98f3-434f-b675-e6a89520fe88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:53.993315 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19614250-98f3-434f-b675-e6a89520fe88 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:53.993603 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19614250-98f3-434f-b675-e6a89520fe88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:53.993885 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19614250-98f3-434f-b675-e6a89520fe88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:53.996387 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19614250-98f3-434f-b675-e6a89520fe88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:53.999613 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19614250-98f3-434f-b675-e6a89520fe88 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:54.012562 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899bb7a5-e160-4413-b04f-6f7d31540850 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:54.024034 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e1e55-8210-4dfd-a265-557486a6d9d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:54.024034 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e1e55-8210-4dfd-a265-557486a6d9d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:54.024034 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e1e55-8210-4dfd-a265-557486a6d9d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:54.024034 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e1e55-8210-4dfd-a265-557486a6d9d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:54.024516 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 980/1962] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:53 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:19:54.024516 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19614250-98f3-434f-b675-e6a89520fe88 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:54.031012 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5e1e55-8210-4dfd-a265-557486a6d9d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:54.032719 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5e1e55-8210-4dfd-a265-557486a6d9d0 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:54.046136 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 983/1963] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:53 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:19:54.047215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19614250-98f3-434f-b675-e6a89520fe88 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:54.059677 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e291fe7-5843-4cb6-86cf-21014ba9777a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:54.060591 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e291fe7-5843-4cb6-86cf-21014ba9777a admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:54.062345 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e291fe7-5843-4cb6-86cf-21014ba9777a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:54.063101 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e1e55-8210-4dfd-a265-557486a6d9d0 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:54.064179 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e291fe7-5843-4cb6-86cf-21014ba9777a admin admin] PATH_INFO: `/v3/projects/b5a82417eaf44a70a3fde784fd544cbf/users/6de784241d2843f0b629f71ad5c8d01b/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:54.083834 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e1e55-8210-4dfd-a265-557486a6d9d0 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:54.111163 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e291fe7-5843-4cb6-86cf-21014ba9777a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5a82417eaf44a70a3fde784fd544cbf, user_id=6de784241d2843f0b629f71ad5c8d01b, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:54.114064 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5723845-bc00-4be5-9180-f15fa3d4b2fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:54.114801 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e291fe7-5843-4cb6-86cf-21014ba9777a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:54.115137 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5723845-bc00-4be5-9180-f15fa3d4b2fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:54.115498 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5723845-bc00-4be5-9180-f15fa3d4b2fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:54.115842 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5723845-bc00-4be5-9180-f15fa3d4b2fd admin admin] PATH_INFO: `/v3/projects/2e0c164b1bca495ead041f096ccf073c/users/3e2ace9766054949b3ef75d15abea37d/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:54.144154 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 981/1964] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:19:54 2022] PUT /identity/v3/projects/b5a82417eaf44a70a3fde784fd544cbf/users/6de784241d2843f0b629f71ad5c8d01b/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:54.157152 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e964ec96-00c1-41ef-989f-c276b5c23cfd None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:54.157778 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e964ec96-00c1-41ef-989f-c276b5c23cfd None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:54.158333 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e964ec96-00c1-41ef-989f-c276b5c23cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:54.181317 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5723845-bc00-4be5-9180-f15fa3d4b2fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e0c164b1bca495ead041f096ccf073c, user_id=3e2ace9766054949b3ef75d15abea37d, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:54.187828 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5723845-bc00-4be5-9180-f15fa3d4b2fd admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:54.203806 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-e964ec96-00c1-41ef-989f-c276b5c23cfd None None] MFA Rules not processed for user `6de784241d2843f0b629f71ad5c8d01b`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:54.206619 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e964ec96-00c1-41ef-989f-c276b5c23cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:54.218572 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 984/1965] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:19:54 2022] PUT /identity/v3/projects/2e0c164b1bca495ead041f096ccf073c/users/3e2ace9766054949b3ef75d15abea37d/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:54.230498 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11900295-a3fe-4fd3-b7c4-476d81d028a4 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:54.230794 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11900295-a3fe-4fd3-b7c4-476d81d028a4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:54.231188 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11900295-a3fe-4fd3-b7c4-476d81d028a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:54.277103 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-11900295-a3fe-4fd3-b7c4-476d81d028a4 None None] MFA Rules not processed for user `3e2ace9766054949b3ef75d15abea37d`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:54.278738 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-11900295-a3fe-4fd3-b7c4-476d81d028a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:54.390233 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 982/1966] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:54.407600 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e964ec96-00c1-41ef-989f-c276b5c23cfd None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:54.409000 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e964ec96-00c1-41ef-989f-c276b5c23cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:54.424734 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 985/1967] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:54 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:19:54.425977 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11900295-a3fe-4fd3-b7c4-476d81d028a4 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:54.472863 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:54.486014 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e964ec96-00c1-41ef-989f-c276b5c23cfd None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:54.499205 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11900295-a3fe-4fd3-b7c4-476d81d028a4 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:54.503152 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3d4d3b-dc76-431b-aa3e-371bd48cc350 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:54.503807 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d4d3b-dc76-431b-aa3e-371bd48cc350 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:54.504090 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d4d3b-dc76-431b-aa3e-371bd48cc350 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:54.504369 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d4d3b-dc76-431b-aa3e-371bd48cc350 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:54.506636 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-be3d4d3b-dc76-431b-aa3e-371bd48cc350 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:54.514135 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:54.556077 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd8818-1c5f-4430-b8e3-fa26fbbb1e2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:54.556747 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd8818-1c5f-4430-b8e3-fa26fbbb1e2b service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:54.557061 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd8818-1c5f-4430-b8e3-fa26fbbb1e2b service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:54.557354 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd8818-1c5f-4430-b8e3-fa26fbbb1e2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:54.597770 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3d4d3b-dc76-431b-aa3e-371bd48cc350 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:54.599539 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3d4d3b-dc76-431b-aa3e-371bd48cc350 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:54.610827 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 986/1968] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:54 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:19:54.611890 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3d4d3b-dc76-431b-aa3e-371bd48cc350 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:54.628561 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acd8818-1c5f-4430-b8e3-fa26fbbb1e2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:54.630214 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acd8818-1c5f-4430-b8e3-fa26fbbb1e2b service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:54.641211 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 983/1969] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:19:54.666875 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd8818-1c5f-4430-b8e3-fa26fbbb1e2b service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:54.728148 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3d4d3b-dc76-431b-aa3e-371bd48cc350 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:54.745751 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd5b7d2-9f63-421d-8c07-b8c43b436650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:54.746448 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd5b7d2-9f63-421d-8c07-b8c43b436650 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:54.746734 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd5b7d2-9f63-421d-8c07-b8c43b436650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:54.747014 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd5b7d2-9f63-421d-8c07-b8c43b436650 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:54.747868 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd5b7d2-9f63-421d-8c07-b8c43b436650 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:54.749819 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd5b7d2-9f63-421d-8c07-b8c43b436650 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:54.817212 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 987/1970] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:19:54 2022] POST /identity/v3/projects => generated 377 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:19:54.818304 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd5b7d2-9f63-421d-8c07-b8c43b436650 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:54.818483 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd8818-1c5f-4430-b8e3-fa26fbbb1e2b service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:54.917653 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd5b7d2-9f63-421d-8c07-b8c43b436650 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:54.923364 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb4c7f6-f387-4f98-aff5-4230bb7e9b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:54.925402 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb4c7f6-f387-4f98-aff5-4230bb7e9b2f admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:54.925402 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb4c7f6-f387-4f98-aff5-4230bb7e9b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:54.925402 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb4c7f6-f387-4f98-aff5-4230bb7e9b2f admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:54.930216 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb4c7f6-f387-4f98-aff5-4230bb7e9b2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:54.932628 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb4c7f6-f387-4f98-aff5-4230bb7e9b2f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:54.957426 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:54.968930 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 988/1971] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:19:54 2022] POST /identity/v3/users => generated 295 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:19:54.970508 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb4c7f6-f387-4f98-aff5-4230bb7e9b2f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:54.997306 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17759c1f-3159-4e0a-9be6-cb07876ae80f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:54.998208 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb4c7f6-f387-4f98-aff5-4230bb7e9b2f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:54.998627 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17759c1f-3159-4e0a-9be6-cb07876ae80f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:54.999657 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17759c1f-3159-4e0a-9be6-cb07876ae80f admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:54.999657 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17759c1f-3159-4e0a-9be6-cb07876ae80f admin admin] PATH_INFO: `/v3/users/6de784241d2843f0b629f71ad5c8d01b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:55.002358 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17759c1f-3159-4e0a-9be6-cb07876ae80f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6de784241d2843f0b629f71ad5c8d01b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:55.003338 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e941fa-1a2b-4274-b8dc-d3ec4faad31c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:55.004910 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e941fa-1a2b-4274-b8dc-d3ec4faad31c admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:55.004910 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e941fa-1a2b-4274-b8dc-d3ec4faad31c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:55.004910 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e941fa-1a2b-4274-b8dc-d3ec4faad31c admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:55.005713 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e941fa-1a2b-4274-b8dc-d3ec4faad31c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:55.007319 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17759c1f-3159-4e0a-9be6-cb07876ae80f admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:55.020565 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e941fa-1a2b-4274-b8dc-d3ec4faad31c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:55.043474 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 989/1972] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:19:55.048747 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e941fa-1a2b-4274-b8dc-d3ec4faad31c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:55.065014 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e941fa-1a2b-4274-b8dc-d3ec4faad31c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:55.070742 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa7e263-1c24-41ba-abf4-33c6d5dbbe37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:55.072198 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa7e263-1c24-41ba-abf4-33c6d5dbbe37 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:55.072664 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa7e263-1c24-41ba-abf4-33c6d5dbbe37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:55.072952 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa7e263-1c24-41ba-abf4-33c6d5dbbe37 admin admin] PATH_INFO: `/v3/projects/d4e74710ed0547e886f0bd869948807a/users/72c239db8e164300809154ee03dd77da/roles/f36e053b442540f08041ac7878760f65` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:55.101818 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:55.104600 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa7e263-1c24-41ba-abf4-33c6d5dbbe37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4e74710ed0547e886f0bd869948807a, user_id=72c239db8e164300809154ee03dd77da, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:55.109891 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa7e263-1c24-41ba-abf4-33c6d5dbbe37 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:55.138052 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-17759c1f-3159-4e0a-9be6-cb07876ae80f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6de784241d2843f0b629f71ad5c8d01b'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:55.152463 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-17759c1f-3159-4e0a-9be6-cb07876ae80f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6de784241d2843f0b629f71ad5c8d01b'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:55.152714 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 990/1973] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:19:55 2022] PUT /identity/v3/projects/d4e74710ed0547e886f0bd869948807a/users/72c239db8e164300809154ee03dd77da/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:55.156223 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-17759c1f-3159-4e0a-9be6-cb07876ae80f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6de784241d2843f0b629f71ad5c8d01b'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:55.157129 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa7e263-1c24-41ba-abf4-33c6d5dbbe37 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:55.159820 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-afa7e263-1c24-41ba-abf4-33c6d5dbbe37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:55.180146 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-17759c1f-3159-4e0a-9be6-cb07876ae80f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6de784241d2843f0b629f71ad5c8d01b'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:55.214492 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-17759c1f-3159-4e0a-9be6-cb07876ae80f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6de784241d2843f0b629f71ad5c8d01b'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:55.233177 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 984/1974] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:19:54 2022] DELETE /identity/v3/users/6de784241d2843f0b629f71ad5c8d01b => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:55.238378 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6e3a0-48a1-4511-ba0c-46b40fb7f7bf None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:55.238758 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6e3a0-48a1-4511-ba0c-46b40fb7f7bf None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:55.239070 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6e3a0-48a1-4511-ba0c-46b40fb7f7bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:55.280717 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-20d6e3a0-48a1-4511-ba0c-46b40fb7f7bf None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:55.284921 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-20d6e3a0-48a1-4511-ba0c-46b40fb7f7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:55.286881 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa7e263-1c24-41ba-abf4-33c6d5dbbe37 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:55.369220 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 985/1975] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:55.369694 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c8a03-1d83-4b73-9170-cc61b8ffb08b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:55.373374 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c8a03-1d83-4b73-9170-cc61b8ffb08b admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:55.373665 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c8a03-1d83-4b73-9170-cc61b8ffb08b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:55.373946 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c8a03-1d83-4b73-9170-cc61b8ffb08b admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:55.374705 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863c8a03-1d83-4b73-9170-cc61b8ffb08b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:55.375639 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d6e3a0-48a1-4511-ba0c-46b40fb7f7bf None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:55.377541 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-20d6e3a0-48a1-4511-ba0c-46b40fb7f7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:55.377990 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863c8a03-1d83-4b73-9170-cc61b8ffb08b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:55.396978 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d6e3a0-48a1-4511-ba0c-46b40fb7f7bf None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:55.402513 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 991/1976] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:55 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:19:55.402513 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c8a03-1d83-4b73-9170-cc61b8ffb08b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:55.402513 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-863c8a03-1d83-4b73-9170-cc61b8ffb08b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:55.406369 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555af541-53b6-4c0e-b4b9-af97a8424305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:55.407360 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555af541-53b6-4c0e-b4b9-af97a8424305 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:55.407710 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555af541-53b6-4c0e-b4b9-af97a8424305 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:55.408056 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555af541-53b6-4c0e-b4b9-af97a8424305 admin admin] PATH_INFO: `/v3/users/80feb91acc33494080a74f1305fe404e` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:55.409482 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555af541-53b6-4c0e-b4b9-af97a8424305 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80feb91acc33494080a74f1305fe404e)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:55.411732 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555af541-53b6-4c0e-b4b9-af97a8424305 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:55.467384 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-555af541-53b6-4c0e-b4b9-af97a8424305 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80feb91acc33494080a74f1305fe404e'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:55.472892 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c8a03-1d83-4b73-9170-cc61b8ffb08b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:55.476755 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-555af541-53b6-4c0e-b4b9-af97a8424305 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80feb91acc33494080a74f1305fe404e'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:55.477597 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-555af541-53b6-4c0e-b4b9-af97a8424305 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80feb91acc33494080a74f1305fe404e'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:55.478445 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e746c0c-7fed-420e-a641-dabeba3a8643 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:55.479255 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e746c0c-7fed-420e-a641-dabeba3a8643 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:55.479529 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e746c0c-7fed-420e-a641-dabeba3a8643 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:55.479807 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e746c0c-7fed-420e-a641-dabeba3a8643 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:55.481839 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7e746c0c-7fed-420e-a641-dabeba3a8643 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:55.488694 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-555af541-53b6-4c0e-b4b9-af97a8424305 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80feb91acc33494080a74f1305fe404e'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:55.501247 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e746c0c-7fed-420e-a641-dabeba3a8643 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:55.503214 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e746c0c-7fed-420e-a641-dabeba3a8643 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:55.506674 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-555af541-53b6-4c0e-b4b9-af97a8424305 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80feb91acc33494080a74f1305fe404e'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:55.516449 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 992/1978] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:19:55.517298 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 986/1978] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:19:55 2022] DELETE /identity/v3/users/80feb91acc33494080a74f1305fe404e => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:55.517928 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e746c0c-7fed-420e-a641-dabeba3a8643 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:55.519380 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfea8b5-ed1b-491d-91f2-874ac54d0f46 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:55.519638 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7e746c0c-7fed-420e-a641-dabeba3a8643 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:55.519750 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfea8b5-ed1b-491d-91f2-874ac54d0f46 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:55.519947 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfea8b5-ed1b-491d-91f2-874ac54d0f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:55.536990 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e746c0c-7fed-420e-a641-dabeba3a8643 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:55.542492 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-adfea8b5-ed1b-491d-91f2-874ac54d0f46 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:55.544223 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-adfea8b5-ed1b-491d-91f2-874ac54d0f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:55.546698 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:55.589546 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:55.634643 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2c3fe8-9cc7-4563-9df0-0af897166477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:55.635855 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c3fe8-9cc7-4563-9df0-0af897166477 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:55.636283 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c3fe8-9cc7-4563-9df0-0af897166477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:55.636873 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c3fe8-9cc7-4563-9df0-0af897166477 admin admin] PATH_INFO: `/v3/projects/d4e74710ed0547e886f0bd869948807a/users/72c239db8e164300809154ee03dd77da/roles/f36e053b442540f08041ac7878760f65` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:55.640914 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2c3fe8-9cc7-4563-9df0-0af897166477 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4e74710ed0547e886f0bd869948807a, user_id=72c239db8e164300809154ee03dd77da, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:55.644154 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2c3fe8-9cc7-4563-9df0-0af897166477 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:55.666082 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 993/1979] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:19:55 2022] PUT /identity/v3/projects/d4e74710ed0547e886f0bd869948807a/users/72c239db8e164300809154ee03dd77da/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:55.676546 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9369dcc-84fc-4178-bfb1-b62fd8667ac5 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:55.677066 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9369dcc-84fc-4178-bfb1-b62fd8667ac5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:55.677894 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9369dcc-84fc-4178-bfb1-b62fd8667ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:55.706055 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 987/1980] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:55.722992 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfea8b5-ed1b-491d-91f2-874ac54d0f46 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:55.724817 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-adfea8b5-ed1b-491d-91f2-874ac54d0f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:55.742214 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d9369dcc-84fc-4178-bfb1-b62fd8667ac5 None None] MFA Rules not processed for user `72c239db8e164300809154ee03dd77da`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:55.743767 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d9369dcc-84fc-4178-bfb1-b62fd8667ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:55.886072 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfea8b5-ed1b-491d-91f2-874ac54d0f46 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:55.903554 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 994/1981] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:55 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:19:55.916819 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9369dcc-84fc-4178-bfb1-b62fd8667ac5 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:55.918289 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d9369dcc-84fc-4178-bfb1-b62fd8667ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:55.997171 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5365c2d6-9063-44d0-9df7-601728e1c3b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:55.997756 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365c2d6-9063-44d0-9df7-601728e1c3b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:55.998307 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365c2d6-9063-44d0-9df7-601728e1c3b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:55.998307 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365c2d6-9063-44d0-9df7-601728e1c3b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:56.032154 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9369dcc-84fc-4178-bfb1-b62fd8667ac5 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:56.037146 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cd7d72-5a9c-4ec8-ae35-8a3afbc0af7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:56.037395 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd7d72-5a9c-4ec8-ae35-8a3afbc0af7e service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:56.037679 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd7d72-5a9c-4ec8-ae35-8a3afbc0af7e service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:56.037958 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd7d72-5a9c-4ec8-ae35-8a3afbc0af7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:56.051847 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cd7d72-5a9c-4ec8-ae35-8a3afbc0af7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:56.053341 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cd7d72-5a9c-4ec8-ae35-8a3afbc0af7e service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:56.066836 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 995/1982] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:55 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:19:56.084363 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cd7d72-5a9c-4ec8-ae35-8a3afbc0af7e service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:56.087325 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-61cd7d72-5a9c-4ec8-ae35-8a3afbc0af7e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:56.105057 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5365c2d6-9063-44d0-9df7-601728e1c3b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:56.106825 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5365c2d6-9063-44d0-9df7-601728e1c3b6 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:56.121413 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 988/1983] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:19:56.185282 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:56.247990 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cd7d72-5a9c-4ec8-ae35-8a3afbc0af7e service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:56.256347 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10555f84-cfff-4db7-8296-9e466c70884d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:56.257533 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10555f84-cfff-4db7-8296-9e466c70884d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:56.257896 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10555f84-cfff-4db7-8296-9e466c70884d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:56.258248 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10555f84-cfff-4db7-8296-9e466c70884d admin admin] PATH_INFO: `/v3/projects/b5a82417eaf44a70a3fde784fd544cbf` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:56.259662 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10555f84-cfff-4db7-8296-9e466c70884d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5a82417eaf44a70a3fde784fd544cbf)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:56.261896 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10555f84-cfff-4db7-8296-9e466c70884d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:56.336065 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-10555f84-cfff-4db7-8296-9e466c70884d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5a82417eaf44a70a3fde784fd544cbf', 'request_id': 'req-10555f84-cfff-4db7-8296-9e466c70884d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:56.348060 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-10555f84-cfff-4db7-8296-9e466c70884d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5a82417eaf44a70a3fde784fd544cbf', 'request_id': 'req-10555f84-cfff-4db7-8296-9e466c70884d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:56.358341 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-10555f84-cfff-4db7-8296-9e466c70884d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5a82417eaf44a70a3fde784fd544cbf', 'request_id': 'req-10555f84-cfff-4db7-8296-9e466c70884d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:56.385526 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-10555f84-cfff-4db7-8296-9e466c70884d admin admin] The token cache is being invalidate because project b5a82417eaf44a70a3fde784fd544cbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:19:56.386204 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-10555f84-cfff-4db7-8296-9e466c70884d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:56.387144 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-10555f84-cfff-4db7-8296-9e466c70884d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:56.389462 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 996/1984] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:19:56 2022] DELETE /identity/v3/projects/b5a82417eaf44a70a3fde784fd544cbf => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:56.395588 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b727af-6060-4454-ad46-d6a90a754d64 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:56.398186 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b727af-6060-4454-ad46-d6a90a754d64 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:56.398490 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b727af-6060-4454-ad46-d6a90a754d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:56.446237 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-18b727af-6060-4454-ad46-d6a90a754d64 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:56.446237 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-18b727af-6060-4454-ad46-d6a90a754d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:56.526425 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:56.633099 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 989/1985] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:56.643062 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf157652-818c-4493-9d17-ebbe5a0b58f7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:56.644657 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf157652-818c-4493-9d17-ebbe5a0b58f7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:56.645081 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf157652-818c-4493-9d17-ebbe5a0b58f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:56.668456 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-bf157652-818c-4493-9d17-ebbe5a0b58f7 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:56.670263 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bf157652-818c-4493-9d17-ebbe5a0b58f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:56.681856 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 997/1986] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:56.685788 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b727af-6060-4454-ad46-d6a90a754d64 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:56.691333 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf157652-818c-4493-9d17-ebbe5a0b58f7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:56.692747 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bf157652-818c-4493-9d17-ebbe5a0b58f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:56.698968 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b727af-6060-4454-ad46-d6a90a754d64 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:56.705391 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b905b6e-e753-42a6-b13e-63e0f3cfcbc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:56.706431 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b905b6e-e753-42a6-b13e-63e0f3cfcbc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:56.706617 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b905b6e-e753-42a6-b13e-63e0f3cfcbc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:56.707398 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b905b6e-e753-42a6-b13e-63e0f3cfcbc5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:56.708206 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b905b6e-e753-42a6-b13e-63e0f3cfcbc5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:19:56.708676 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b905b6e-e753-42a6-b13e-63e0f3cfcbc5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:56.713273 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b905b6e-e753-42a6-b13e-63e0f3cfcbc5 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:56.741245 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 990/1987] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:19:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:19:56.742535 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b905b6e-e753-42a6-b13e-63e0f3cfcbc5 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:56.756590 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b905b6e-e753-42a6-b13e-63e0f3cfcbc5 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:56.762677 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab02714-1a54-480a-9db9-87de5df1695c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:56.763362 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab02714-1a54-480a-9db9-87de5df1695c admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:56.763650 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab02714-1a54-480a-9db9-87de5df1695c admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:56.763936 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab02714-1a54-480a-9db9-87de5df1695c admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:56.764793 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab02714-1a54-480a-9db9-87de5df1695c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:56.768023 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab02714-1a54-480a-9db9-87de5df1695c admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:56.796915 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf157652-818c-4493-9d17-ebbe5a0b58f7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:56.804167 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c469c6f-387e-4944-a2fa-5ec163eeca87 tempest-VolumeQuotaClassesTest-87490277 tempest-VolumeQuotaClassesTest-87490277-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c239db8e164300809154ee03dd77da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4e74710ed0547e886f0bd869948807a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:56.805105 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c469c6f-387e-4944-a2fa-5ec163eeca87 tempest-VolumeQuotaClassesTest-87490277 tempest-VolumeQuotaClassesTest-87490277-project-admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:56.805521 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c469c6f-387e-4944-a2fa-5ec163eeca87 tempest-VolumeQuotaClassesTest-87490277 tempest-VolumeQuotaClassesTest-87490277-project-admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:56.806221 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c469c6f-387e-4944-a2fa-5ec163eeca87 tempest-VolumeQuotaClassesTest-87490277 tempest-VolumeQuotaClassesTest-87490277-project-admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:56.807307 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c469c6f-387e-4944-a2fa-5ec163eeca87 tempest-VolumeQuotaClassesTest-87490277 tempest-VolumeQuotaClassesTest-87490277-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:19:56.807709 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c469c6f-387e-4944-a2fa-5ec163eeca87 tempest-VolumeQuotaClassesTest-87490277 tempest-VolumeQuotaClassesTest-87490277-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:56.809501 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c469c6f-387e-4944-a2fa-5ec163eeca87 tempest-VolumeQuotaClassesTest-87490277 tempest-VolumeQuotaClassesTest-87490277-project-admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:56.815233 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 991/1988] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:19:56 2022] POST /identity/v3/projects => generated 361 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:19:56.817363 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab02714-1a54-480a-9db9-87de5df1695c admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:56.836312 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 998/1989] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:19:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:19:56.841988 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c469c6f-387e-4944-a2fa-5ec163eeca87 tempest-VolumeQuotaClassesTest-87490277 tempest-VolumeQuotaClassesTest-87490277-project-admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:56.856913 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c469c6f-387e-4944-a2fa-5ec163eeca87 tempest-VolumeQuotaClassesTest-87490277 tempest-VolumeQuotaClassesTest-87490277-project-admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:56.919149 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab02714-1a54-480a-9db9-87de5df1695c admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:56.919963 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:56.925368 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3a3972-0f3b-424c-8ee6-a89fbbf43ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:56.926188 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3a3972-0f3b-424c-8ee6-a89fbbf43ff0 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:56.926752 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3a3972-0f3b-424c-8ee6-a89fbbf43ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:56.927212 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3a3972-0f3b-424c-8ee6-a89fbbf43ff0 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:56.928092 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3a3972-0f3b-424c-8ee6-a89fbbf43ff0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:56.930917 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3a3972-0f3b-424c-8ee6-a89fbbf43ff0 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:56.940839 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60de729c-9ce1-4880-bea6-e378513c3795 tempest-VolumeQuotaClassesTest-87490277 tempest-VolumeQuotaClassesTest-87490277-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c239db8e164300809154ee03dd77da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4e74710ed0547e886f0bd869948807a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:56.943120 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60de729c-9ce1-4880-bea6-e378513c3795 tempest-VolumeQuotaClassesTest-87490277 tempest-VolumeQuotaClassesTest-87490277-project-admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:56.949315 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60de729c-9ce1-4880-bea6-e378513c3795 tempest-VolumeQuotaClassesTest-87490277 tempest-VolumeQuotaClassesTest-87490277-project-admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:56.951791 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60de729c-9ce1-4880-bea6-e378513c3795 tempest-VolumeQuotaClassesTest-87490277 tempest-VolumeQuotaClassesTest-87490277-project-admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:56.954613 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60de729c-9ce1-4880-bea6-e378513c3795 tempest-VolumeQuotaClassesTest-87490277 tempest-VolumeQuotaClassesTest-87490277-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:56.959018 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60de729c-9ce1-4880-bea6-e378513c3795 tempest-VolumeQuotaClassesTest-87490277 tempest-VolumeQuotaClassesTest-87490277-project-admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:56.959693 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 992/1990] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:19:56 2022] POST /identity/v3/users => generated 281 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:19:56.964924 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3a3972-0f3b-424c-8ee6-a89fbbf43ff0 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:56.979202 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3a3972-0f3b-424c-8ee6-a89fbbf43ff0 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:56.986327 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6e9d68-200b-43b0-9fb8-0ef1e5f187f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:56.987167 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e9d68-200b-43b0-9fb8-0ef1e5f187f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:56.987494 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e9d68-200b-43b0-9fb8-0ef1e5f187f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:56.988388 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e9d68-200b-43b0-9fb8-0ef1e5f187f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:56.988514 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6e9d68-200b-43b0-9fb8-0ef1e5f187f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:56.990322 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6e9d68-200b-43b0-9fb8-0ef1e5f187f8 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:57.004896 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 999/1991] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:19:56 2022] POST /identity/v3/projects => generated 355 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:19:57.006650 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60de729c-9ce1-4880-bea6-e378513c3795 tempest-VolumeQuotaClassesTest-87490277 tempest-VolumeQuotaClassesTest-87490277-project-admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:57.008463 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-60de729c-9ce1-4880-bea6-e378513c3795 tempest-VolumeQuotaClassesTest-87490277 tempest-VolumeQuotaClassesTest-87490277-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:57.010435 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 993/1992] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:56 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:19:57.014026 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6e9d68-200b-43b0-9fb8-0ef1e5f187f8 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:57.034352 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6e9d68-200b-43b0-9fb8-0ef1e5f187f8 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:57.131561 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8a570-fdc2-4522-9a47-d68ff737941c tempest-VolumeQuotaClassesTest-87490277 tempest-VolumeQuotaClassesTest-87490277-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c239db8e164300809154ee03dd77da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4e74710ed0547e886f0bd869948807a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:57.133004 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8a570-fdc2-4522-9a47-d68ff737941c tempest-VolumeQuotaClassesTest-87490277 tempest-VolumeQuotaClassesTest-87490277-project-admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:57.133466 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8a570-fdc2-4522-9a47-d68ff737941c tempest-VolumeQuotaClassesTest-87490277 tempest-VolumeQuotaClassesTest-87490277-project-admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:57.133883 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8a570-fdc2-4522-9a47-d68ff737941c tempest-VolumeQuotaClassesTest-87490277 tempest-VolumeQuotaClassesTest-87490277-project-admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:57.134637 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e8a570-fdc2-4522-9a47-d68ff737941c tempest-VolumeQuotaClassesTest-87490277 tempest-VolumeQuotaClassesTest-87490277-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:19:57.134952 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e8a570-fdc2-4522-9a47-d68ff737941c tempest-VolumeQuotaClassesTest-87490277 tempest-VolumeQuotaClassesTest-87490277-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:57.137016 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e8a570-fdc2-4522-9a47-d68ff737941c tempest-VolumeQuotaClassesTest-87490277 tempest-VolumeQuotaClassesTest-87490277-project-admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:57.154209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 994/1993] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:19:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:19:57.155204 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8a570-fdc2-4522-9a47-d68ff737941c tempest-VolumeQuotaClassesTest-87490277 tempest-VolumeQuotaClassesTest-87490277-project-admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:57.156065 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60de729c-9ce1-4880-bea6-e378513c3795 tempest-VolumeQuotaClassesTest-87490277 tempest-VolumeQuotaClassesTest-87490277-project-admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:57.161920 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85889a9-c943-40b3-aa06-ca0035b6022b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:57.162957 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85889a9-c943-40b3-aa06-ca0035b6022b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:57.163575 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85889a9-c943-40b3-aa06-ca0035b6022b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:57.163868 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85889a9-c943-40b3-aa06-ca0035b6022b admin admin] PATH_INFO: `/v3/projects/24b88a9e7afd4ac0bd003bb3f0f283f8` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:57.165245 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85889a9-c943-40b3-aa06-ca0035b6022b admin admin] RBAC: Authorizing `identity:delete_project(project_id=24b88a9e7afd4ac0bd003bb3f0f283f8)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:57.167029 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:57.168192 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85889a9-c943-40b3-aa06-ca0035b6022b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:57.173929 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8a570-fdc2-4522-9a47-d68ff737941c tempest-VolumeQuotaClassesTest-87490277 tempest-VolumeQuotaClassesTest-87490277-project-admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:57.182240 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a35f05b-ba43-4b1f-97d2-f25551051a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:57.183028 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a35f05b-ba43-4b1f-97d2-f25551051a53 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:57.183403 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a35f05b-ba43-4b1f-97d2-f25551051a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:57.183759 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a35f05b-ba43-4b1f-97d2-f25551051a53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:57.185622 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a35f05b-ba43-4b1f-97d2-f25551051a53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:57.187334 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a35f05b-ba43-4b1f-97d2-f25551051a53 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:57.203202 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 995/1994] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:57 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:19:57.208352 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a35f05b-ba43-4b1f-97d2-f25551051a53 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:57.227755 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a35f05b-ba43-4b1f-97d2-f25551051a53 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:57.236278 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fca635-2c5b-4fed-8c8d-4061863442ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:57.239439 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fca635-2c5b-4fed-8c8d-4061863442ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:57.239977 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fca635-2c5b-4fed-8c8d-4061863442ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:57.241774 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fca635-2c5b-4fed-8c8d-4061863442ea admin admin] PATH_INFO: `/v3/projects/41ed734c32544b988ee1df0486a10d37/users/e8bb3aaa40644dbf819e3feba08203cb/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:57.274417 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fca635-2c5b-4fed-8c8d-4061863442ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=41ed734c32544b988ee1df0486a10d37, user_id=e8bb3aaa40644dbf819e3feba08203cb, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:57.277201 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fca635-2c5b-4fed-8c8d-4061863442ea admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:57.288918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f85889a9-c943-40b3-aa06-ca0035b6022b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24b88a9e7afd4ac0bd003bb3f0f283f8', 'request_id': 'req-f85889a9-c943-40b3-aa06-ca0035b6022b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:57.309163 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f85889a9-c943-40b3-aa06-ca0035b6022b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24b88a9e7afd4ac0bd003bb3f0f283f8', 'request_id': 'req-f85889a9-c943-40b3-aa06-ca0035b6022b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:57.321628 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 996/1995] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:19:57 2022] PUT /identity/v3/projects/41ed734c32544b988ee1df0486a10d37/users/e8bb3aaa40644dbf819e3feba08203cb/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:57.321628 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fca635-2c5b-4fed-8c8d-4061863442ea admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:57.326676 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f85889a9-c943-40b3-aa06-ca0035b6022b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24b88a9e7afd4ac0bd003bb3f0f283f8', 'request_id': 'req-f85889a9-c943-40b3-aa06-ca0035b6022b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:57.349088 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f85889a9-c943-40b3-aa06-ca0035b6022b admin admin] The token cache is being invalidate because project 24b88a9e7afd4ac0bd003bb3f0f283f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:19:57.349088 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f85889a9-c943-40b3-aa06-ca0035b6022b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:57.350146 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f85889a9-c943-40b3-aa06-ca0035b6022b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:57.353211 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1000/1996] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:19:57 2022] DELETE /identity/v3/projects/24b88a9e7afd4ac0bd003bb3f0f283f8 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:57.354744 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71152fb-a2d5-4251-b934-099fd41da6fd None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:57.355040 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71152fb-a2d5-4251-b934-099fd41da6fd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:57.355318 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71152fb-a2d5-4251-b934-099fd41da6fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:57.401617 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b71152fb-a2d5-4251-b934-099fd41da6fd None None] MFA Rules not processed for user `e8bb3aaa40644dbf819e3feba08203cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:57.403279 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b71152fb-a2d5-4251-b934-099fd41da6fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:57.444033 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fca635-2c5b-4fed-8c8d-4061863442ea admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:57.450295 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f877f-e7f3-4838-a6de-a20035d18c32 tempest-VolumeQuotaClassesTest-87490277 tempest-VolumeQuotaClassesTest-87490277-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c239db8e164300809154ee03dd77da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd4e74710ed0547e886f0bd869948807a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:57.451571 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f877f-e7f3-4838-a6de-a20035d18c32 tempest-VolumeQuotaClassesTest-87490277 tempest-VolumeQuotaClassesTest-87490277-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:57.451933 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f877f-e7f3-4838-a6de-a20035d18c32 tempest-VolumeQuotaClassesTest-87490277 tempest-VolumeQuotaClassesTest-87490277-project-admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:57.452283 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f877f-e7f3-4838-a6de-a20035d18c32 tempest-VolumeQuotaClassesTest-87490277 tempest-VolumeQuotaClassesTest-87490277-project-admin] PATH_INFO: `/v3/projects/0965552f752d407cb38952c8f37aa91e` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:57.453658 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50f877f-e7f3-4838-a6de-a20035d18c32 tempest-VolumeQuotaClassesTest-87490277 tempest-VolumeQuotaClassesTest-87490277-project-admin] RBAC: Authorizing `identity:delete_project(project_id=0965552f752d407cb38952c8f37aa91e)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:57.455769 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50f877f-e7f3-4838-a6de-a20035d18c32 tempest-VolumeQuotaClassesTest-87490277 tempest-VolumeQuotaClassesTest-87490277-project-admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:57.480412 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:57.495645 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1001/1997] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:57 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:57.496336 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71152fb-a2d5-4251-b934-099fd41da6fd None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:57.503539 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b71152fb-a2d5-4251-b934-099fd41da6fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:57.551317 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a50f877f-e7f3-4838-a6de-a20035d18c32 tempest-VolumeQuotaClassesTest-87490277 tempest-VolumeQuotaClassesTest-87490277-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0965552f752d407cb38952c8f37aa91e', 'request_id': 'req-a50f877f-e7f3-4838-a6de-a20035d18c32'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:57.570736 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a50f877f-e7f3-4838-a6de-a20035d18c32 tempest-VolumeQuotaClassesTest-87490277 tempest-VolumeQuotaClassesTest-87490277-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0965552f752d407cb38952c8f37aa91e', 'request_id': 'req-a50f877f-e7f3-4838-a6de-a20035d18c32'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:57.590065 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a50f877f-e7f3-4838-a6de-a20035d18c32 tempest-VolumeQuotaClassesTest-87490277 tempest-VolumeQuotaClassesTest-87490277-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0965552f752d407cb38952c8f37aa91e', 'request_id': 'req-a50f877f-e7f3-4838-a6de-a20035d18c32'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:57.608816 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a50f877f-e7f3-4838-a6de-a20035d18c32 tempest-VolumeQuotaClassesTest-87490277 tempest-VolumeQuotaClassesTest-87490277-project-admin] The token cache is being invalidate because project 0965552f752d407cb38952c8f37aa91e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:19:57.609603 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a50f877f-e7f3-4838-a6de-a20035d18c32 tempest-VolumeQuotaClassesTest-87490277 tempest-VolumeQuotaClassesTest-87490277-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:57.612254 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a50f877f-e7f3-4838-a6de-a20035d18c32 tempest-VolumeQuotaClassesTest-87490277 tempest-VolumeQuotaClassesTest-87490277-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:57.614405 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 997/1998] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:19:57 2022] DELETE /identity/v3/projects/0965552f752d407cb38952c8f37aa91e => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:57.615696 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f877f-e7f3-4838-a6de-a20035d18c32 tempest-VolumeQuotaClassesTest-87490277 tempest-VolumeQuotaClassesTest-87490277-project-admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:57.619366 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a50f877f-e7f3-4838-a6de-a20035d18c32 tempest-VolumeQuotaClassesTest-87490277 tempest-VolumeQuotaClassesTest-87490277-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:57.629758 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71152fb-a2d5-4251-b934-099fd41da6fd None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:57.759562 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f877f-e7f3-4838-a6de-a20035d18c32 tempest-VolumeQuotaClassesTest-87490277 tempest-VolumeQuotaClassesTest-87490277-project-admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:57.762086 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b96f79-9d53-41ef-aea3-5b71ac8bf66f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:57.762785 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b96f79-9d53-41ef-aea3-5b71ac8bf66f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:57.763616 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b96f79-9d53-41ef-aea3-5b71ac8bf66f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:57.763899 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b96f79-9d53-41ef-aea3-5b71ac8bf66f admin admin] PATH_INFO: `/v3/users/c3127dea8f084a4c8e59321507192892` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:57.766777 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b96f79-9d53-41ef-aea3-5b71ac8bf66f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3127dea8f084a4c8e59321507192892)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:57.768796 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a0e23f-bde9-48ff-94bd-61c232b6e528 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:57.770495 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a0e23f-bde9-48ff-94bd-61c232b6e528 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:57.770692 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a0e23f-bde9-48ff-94bd-61c232b6e528 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:57.770962 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a0e23f-bde9-48ff-94bd-61c232b6e528 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:57.772317 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b96f79-9d53-41ef-aea3-5b71ac8bf66f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:57.777026 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f3a0e23f-bde9-48ff-94bd-61c232b6e528 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:57.813683 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:57.869465 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-51b96f79-9d53-41ef-aea3-5b71ac8bf66f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3127dea8f084a4c8e59321507192892'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:57.870252 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-51b96f79-9d53-41ef-aea3-5b71ac8bf66f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3127dea8f084a4c8e59321507192892'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:57.886385 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-51b96f79-9d53-41ef-aea3-5b71ac8bf66f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3127dea8f084a4c8e59321507192892'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:57.909962 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-51b96f79-9d53-41ef-aea3-5b71ac8bf66f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3127dea8f084a4c8e59321507192892'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:57.910819 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-51b96f79-9d53-41ef-aea3-5b71ac8bf66f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3127dea8f084a4c8e59321507192892'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:57.931902 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1002/1999] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:19:57 2022] DELETE /identity/v3/users/c3127dea8f084a4c8e59321507192892 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:57.932200 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a0e23f-bde9-48ff-94bd-61c232b6e528 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:57.933816 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a0e23f-bde9-48ff-94bd-61c232b6e528 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:57.955724 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 998/2000] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:57 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:19:57.973548 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b96f79-9d53-41ef-aea3-5b71ac8bf66f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:57.980338 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-51b96f79-9d53-41ef-aea3-5b71ac8bf66f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:58.031148 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:58.144790 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b96f79-9d53-41ef-aea3-5b71ac8bf66f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:58.158907 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a0e23f-bde9-48ff-94bd-61c232b6e528 service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:58.160979 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f3a0e23f-bde9-48ff-94bd-61c232b6e528 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:58.168245 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e652e83-a0c5-4566-8c2a-c4c065630757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:58.170820 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e652e83-a0c5-4566-8c2a-c4c065630757 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:58.171464 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e652e83-a0c5-4566-8c2a-c4c065630757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:58.173352 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e652e83-a0c5-4566-8c2a-c4c065630757 admin admin] PATH_INFO: `/v3/users/e8bb3aaa40644dbf819e3feba08203cb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:58.176440 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e652e83-a0c5-4566-8c2a-c4c065630757 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8bb3aaa40644dbf819e3feba08203cb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:58.185650 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e652e83-a0c5-4566-8c2a-c4c065630757 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:58.206668 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a0e23f-bde9-48ff-94bd-61c232b6e528 service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:58.229880 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fe260a-5a15-4d76-8c0a-d0de321ff3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:58.231081 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe260a-5a15-4d76-8c0a-d0de321ff3b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:58.231580 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe260a-5a15-4d76-8c0a-d0de321ff3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:58.232055 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe260a-5a15-4d76-8c0a-d0de321ff3b8 admin admin] PATH_INFO: `/v3/users/3e2ace9766054949b3ef75d15abea37d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:58.234353 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fe260a-5a15-4d76-8c0a-d0de321ff3b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e2ace9766054949b3ef75d15abea37d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:58.236543 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fe260a-5a15-4d76-8c0a-d0de321ff3b8 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:58.317137 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0e652e83-a0c5-4566-8c2a-c4c065630757 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8bb3aaa40644dbf819e3feba08203cb'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:58.318836 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0e652e83-a0c5-4566-8c2a-c4c065630757 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8bb3aaa40644dbf819e3feba08203cb'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:58.335818 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0e652e83-a0c5-4566-8c2a-c4c065630757 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8bb3aaa40644dbf819e3feba08203cb'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:58.347796 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-42fe260a-5a15-4d76-8c0a-d0de321ff3b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e2ace9766054949b3ef75d15abea37d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:58.357251 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0e652e83-a0c5-4566-8c2a-c4c065630757 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8bb3aaa40644dbf819e3feba08203cb'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:58.358146 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0e652e83-a0c5-4566-8c2a-c4c065630757 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8bb3aaa40644dbf819e3feba08203cb'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:58.366562 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-42fe260a-5a15-4d76-8c0a-d0de321ff3b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e2ace9766054949b3ef75d15abea37d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:58.367360 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-42fe260a-5a15-4d76-8c0a-d0de321ff3b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e2ace9766054949b3ef75d15abea37d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:58.402022 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-42fe260a-5a15-4d76-8c0a-d0de321ff3b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e2ace9766054949b3ef75d15abea37d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:58.406280 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1003/2001] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:19:57 2022] DELETE /identity/v3/users/e8bb3aaa40644dbf819e3feba08203cb => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:58.412019 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ab1ff-3dc2-46e5-a107-1d885dc5ca88 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:58.412747 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ab1ff-3dc2-46e5-a107-1d885dc5ca88 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:58.414839 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ab1ff-3dc2-46e5-a107-1d885dc5ca88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:58.427037 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-42fe260a-5a15-4d76-8c0a-d0de321ff3b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e2ace9766054949b3ef75d15abea37d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:58.440775 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 999/2002] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:19:58 2022] DELETE /identity/v3/users/3e2ace9766054949b3ef75d15abea37d => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:58.446450 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b17d69-e511-42ad-b224-6f968dde4d1c None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:58.446750 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b17d69-e511-42ad-b224-6f968dde4d1c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:58.447034 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b17d69-e511-42ad-b224-6f968dde4d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:58.462189 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4e4ab1ff-3dc2-46e5-a107-1d885dc5ca88 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:58.463533 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4e4ab1ff-3dc2-46e5-a107-1d885dc5ca88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:58.484331 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-73b17d69-e511-42ad-b224-6f968dde4d1c None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:58.485883 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-73b17d69-e511-42ad-b224-6f968dde4d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:58.596687 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1004/2003] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:58.606591 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4ab1ff-3dc2-46e5-a107-1d885dc5ca88 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:58.608031 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4e4ab1ff-3dc2-46e5-a107-1d885dc5ca88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:58.650651 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1000/2004] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:58.699860 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b17d69-e511-42ad-b224-6f968dde4d1c None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:58.707890 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-73b17d69-e511-42ad-b224-6f968dde4d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:58.735817 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:58.762242 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4ab1ff-3dc2-46e5-a107-1d885dc5ca88 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:58.768663 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3233bb0-6476-48c4-a0a9-ebe4caaebfbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:58.769415 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3233bb0-6476-48c4-a0a9-ebe4caaebfbd service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:58.769699 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3233bb0-6476-48c4-a0a9-ebe4caaebfbd service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:58.769980 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3233bb0-6476-48c4-a0a9-ebe4caaebfbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:58.781295 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3233bb0-6476-48c4-a0a9-ebe4caaebfbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:58.782743 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3233bb0-6476-48c4-a0a9-ebe4caaebfbd service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:58.797112 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:58.800500 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1005/2005] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:19:58.831063 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3233bb0-6476-48c4-a0a9-ebe4caaebfbd service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:58.836140 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c3233bb0-6476-48c4-a0a9-ebe4caaebfbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:58.858649 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3233bb0-6476-48c4-a0a9-ebe4caaebfbd service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:58.860757 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b17d69-e511-42ad-b224-6f968dde4d1c None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:58.864911 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26962c54-602a-4d4b-8da7-010b6d33a255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:58.867006 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26962c54-602a-4d4b-8da7-010b6d33a255 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:58.867455 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26962c54-602a-4d4b-8da7-010b6d33a255 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:58.867880 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26962c54-602a-4d4b-8da7-010b6d33a255 admin admin] PATH_INFO: `/v3/projects/e97cea692c284df1b938ee46bda369b0` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:58.869310 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26962c54-602a-4d4b-8da7-010b6d33a255 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e97cea692c284df1b938ee46bda369b0)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:58.871594 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26962c54-602a-4d4b-8da7-010b6d33a255 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:58.872341 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e05b6dc-5567-4731-80df-34e3a0b59d02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:58.873031 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e05b6dc-5567-4731-80df-34e3a0b59d02 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:58.873341 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e05b6dc-5567-4731-80df-34e3a0b59d02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:58.873625 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e05b6dc-5567-4731-80df-34e3a0b59d02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:58.899989 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e05b6dc-5567-4731-80df-34e3a0b59d02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:58.901503 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e05b6dc-5567-4731-80df-34e3a0b59d02 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:58.931369 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1001/2006] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:19:58.935161 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-26962c54-602a-4d4b-8da7-010b6d33a255 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e97cea692c284df1b938ee46bda369b0', 'request_id': 'req-26962c54-602a-4d4b-8da7-010b6d33a255'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:58.943095 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-26962c54-602a-4d4b-8da7-010b6d33a255 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e97cea692c284df1b938ee46bda369b0', 'request_id': 'req-26962c54-602a-4d4b-8da7-010b6d33a255'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:58.950808 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-26962c54-602a-4d4b-8da7-010b6d33a255 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e97cea692c284df1b938ee46bda369b0', 'request_id': 'req-26962c54-602a-4d4b-8da7-010b6d33a255'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:58.972482 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-26962c54-602a-4d4b-8da7-010b6d33a255 admin admin] The token cache is being invalidate because project e97cea692c284df1b938ee46bda369b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:19:58.972908 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-26962c54-602a-4d4b-8da7-010b6d33a255 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:58.973669 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-26962c54-602a-4d4b-8da7-010b6d33a255 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:58.979945 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1006/2007] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:19:58 2022] DELETE /identity/v3/projects/e97cea692c284df1b938ee46bda369b0 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:58.981714 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee148dd-8384-4b5b-bb3d-d673c85e34fa None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:58.981714 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee148dd-8384-4b5b-bb3d-d673c85e34fa None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:58.981942 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee148dd-8384-4b5b-bb3d-d673c85e34fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:59.018945 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-fee148dd-8384-4b5b-bb3d-d673c85e34fa None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:59.020453 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-fee148dd-8384-4b5b-bb3d-d673c85e34fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:59.163173 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2431346-811f-40b3-a623-c575d84b701a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:59.163700 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2431346-811f-40b3-a623-c575d84b701a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:59.164124 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2431346-811f-40b3-a623-c575d84b701a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:59.168413 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1002/2008] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:59.200496 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a2431346-811f-40b3-a623-c575d84b701a None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:59.212219 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a2431346-811f-40b3-a623-c575d84b701a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:59.242456 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee148dd-8384-4b5b-bb3d-d673c85e34fa None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:59.247300 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1007/2009] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:19:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:59.254679 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee148dd-8384-4b5b-bb3d-d673c85e34fa None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:59.266387 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed40f553-18aa-48ea-89f3-60f86406a0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:59.267172 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed40f553-18aa-48ea-89f3-60f86406a0a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:59.267467 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed40f553-18aa-48ea-89f3-60f86406a0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:59.267961 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed40f553-18aa-48ea-89f3-60f86406a0a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:59.268770 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed40f553-18aa-48ea-89f3-60f86406a0a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:19:59.269160 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed40f553-18aa-48ea-89f3-60f86406a0a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:59.270974 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed40f553-18aa-48ea-89f3-60f86406a0a4 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:59.294823 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2431346-811f-40b3-a623-c575d84b701a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:59.309517 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1003/2010] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:19:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:19:59.311358 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2431346-811f-40b3-a623-c575d84b701a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:59.316340 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdc9a0d-5a80-4e50-9555-c576efb1ddcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:59.317018 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc9a0d-5a80-4e50-9555-c576efb1ddcc admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:59.317734 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc9a0d-5a80-4e50-9555-c576efb1ddcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:59.318027 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc9a0d-5a80-4e50-9555-c576efb1ddcc admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:59.318742 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdc9a0d-5a80-4e50-9555-c576efb1ddcc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:59.326041 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdc9a0d-5a80-4e50-9555-c576efb1ddcc admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:59.401499 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1008/2011] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:19:59 2022] POST /identity/v3/projects => generated 383 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:19:59.406585 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed40f553-18aa-48ea-89f3-60f86406a0a4 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:59.425199 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed40f553-18aa-48ea-89f3-60f86406a0a4 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:59.613553 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de12990-00c4-4260-93e0-43e4fdc503cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:59.614367 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de12990-00c4-4260-93e0-43e4fdc503cb admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:59.615174 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de12990-00c4-4260-93e0-43e4fdc503cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:59.615174 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de12990-00c4-4260-93e0-43e4fdc503cb admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:59.618314 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de12990-00c4-4260-93e0-43e4fdc503cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:59.620646 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de12990-00c4-4260-93e0-43e4fdc503cb admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:59.676141 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1004/2012] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:19:59 2022] POST /identity/v3/users => generated 292 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:19:59.680872 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdc9a0d-5a80-4e50-9555-c576efb1ddcc admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:59.707385 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdc9a0d-5a80-4e50-9555-c576efb1ddcc admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:59.730406 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8baf4d5-5a11-4d24-855e-40a32b561b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:59.732554 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8baf4d5-5a11-4d24-855e-40a32b561b1f admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:59.734765 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8baf4d5-5a11-4d24-855e-40a32b561b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:59.735167 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8baf4d5-5a11-4d24-855e-40a32b561b1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:59.736870 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8baf4d5-5a11-4d24-855e-40a32b561b1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:59.759627 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8baf4d5-5a11-4d24-855e-40a32b561b1f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:59.830549 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1009/2013] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:59 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:19:59.835007 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de12990-00c4-4260-93e0-43e4fdc503cb admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:59.852065 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de12990-00c4-4260-93e0-43e4fdc503cb admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:59.870065 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced48b8f-d68c-4ed6-8fff-ceeda2815297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:59.872067 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced48b8f-d68c-4ed6-8fff-ceeda2815297 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:59.872573 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced48b8f-d68c-4ed6-8fff-ceeda2815297 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:59.876165 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced48b8f-d68c-4ed6-8fff-ceeda2815297 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:59.877565 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced48b8f-d68c-4ed6-8fff-ceeda2815297 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:59.893833 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced48b8f-d68c-4ed6-8fff-ceeda2815297 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:59.929510 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1005/2014] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:19:59 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:19:59.931740 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8baf4d5-5a11-4d24-855e-40a32b561b1f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:59.943629 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8baf4d5-5a11-4d24-855e-40a32b561b1f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:59.949783 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5319aed5-a479-491e-aa55-165620c095c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:59.951671 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5319aed5-a479-491e-aa55-165620c095c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:59.952131 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5319aed5-a479-491e-aa55-165620c095c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:59.952588 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5319aed5-a479-491e-aa55-165620c095c6 admin admin] PATH_INFO: `/v3/projects/a58aafa018954fd4937d70682d6dc780/users/290f50ecfbff4c98b0c0ac8661fdbadd/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:59.975031 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5319aed5-a479-491e-aa55-165620c095c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a58aafa018954fd4937d70682d6dc780, user_id=290f50ecfbff4c98b0c0ac8661fdbadd, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:59.978647 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5319aed5-a479-491e-aa55-165620c095c6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:00.011270 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121c78e8-ebb2-4924-9b53-aa82716b0015 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:00.011626 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121c78e8-ebb2-4924-9b53-aa82716b0015 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:00.011921 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121c78e8-ebb2-4924-9b53-aa82716b0015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:00.013348 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1010/2015] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:19:59 2022] PUT /identity/v3/projects/a58aafa018954fd4937d70682d6dc780/users/290f50ecfbff4c98b0c0ac8661fdbadd/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:20:00.049581 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-121c78e8-ebb2-4924-9b53-aa82716b0015 None None] MFA Rules not processed for user `290f50ecfbff4c98b0c0ac8661fdbadd`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:00.051246 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-121c78e8-ebb2-4924-9b53-aa82716b0015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:00.094541 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:00.225431 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1006/2016] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:20:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:20:00.229775 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5319aed5-a479-491e-aa55-165620c095c6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:00.364784 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5319aed5-a479-491e-aa55-165620c095c6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:00.371697 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0fdc91-0eda-4d5a-9502-d8816c4ac348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:00.372727 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0fdc91-0eda-4d5a-9502-d8816c4ac348 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:00.373169 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0fdc91-0eda-4d5a-9502-d8816c4ac348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:00.374190 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0fdc91-0eda-4d5a-9502-d8816c4ac348 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:00.375002 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0fdc91-0eda-4d5a-9502-d8816c4ac348 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:00.377190 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0fdc91-0eda-4d5a-9502-d8816c4ac348 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:00.396283 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:00.463169 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1011/2017] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:20:00 2022] POST /identity/v3/projects => generated 385 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:20:00.467542 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121c78e8-ebb2-4924-9b53-aa82716b0015 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:00.473890 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0fdc91-0eda-4d5a-9502-d8816c4ac348 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:00.476443 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7f0fdc91-0eda-4d5a-9502-d8816c4ac348 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:00.492126 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121c78e8-ebb2-4924-9b53-aa82716b0015 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:00.621027 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0fdc91-0eda-4d5a-9502-d8816c4ac348 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:00.626526 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c109f707-0198-402e-adb0-cb5b4faaceac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:00.627992 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c109f707-0198-402e-adb0-cb5b4faaceac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:00.628284 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c109f707-0198-402e-adb0-cb5b4faaceac admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:00.628566 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c109f707-0198-402e-adb0-cb5b4faaceac admin admin] PATH_INFO: `/v3/projects/2e0c164b1bca495ead041f096ccf073c` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:00.629819 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c109f707-0198-402e-adb0-cb5b4faaceac admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e0c164b1bca495ead041f096ccf073c)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:00.632862 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c109f707-0198-402e-adb0-cb5b4faaceac admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:00.685950 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4b184-ab5a-42cf-9c5a-8f87d21c0c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:00.688518 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4b184-ab5a-42cf-9c5a-8f87d21c0c61 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:00.689203 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4b184-ab5a-42cf-9c5a-8f87d21c0c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:00.689452 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4b184-ab5a-42cf-9c5a-8f87d21c0c61 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:00.691943 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda4b184-ab5a-42cf-9c5a-8f87d21c0c61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:00.696153 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda4b184-ab5a-42cf-9c5a-8f87d21c0c61 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:00.719892 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c109f707-0198-402e-adb0-cb5b4faaceac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e0c164b1bca495ead041f096ccf073c', 'request_id': 'req-c109f707-0198-402e-adb0-cb5b4faaceac'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:00.740435 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c109f707-0198-402e-adb0-cb5b4faaceac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e0c164b1bca495ead041f096ccf073c', 'request_id': 'req-c109f707-0198-402e-adb0-cb5b4faaceac'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:00.746335 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1007/2018] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:20:00 2022] POST /identity/v3/users => generated 299 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:20:00.751260 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4b184-ab5a-42cf-9c5a-8f87d21c0c61 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:00.752602 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c109f707-0198-402e-adb0-cb5b4faaceac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e0c164b1bca495ead041f096ccf073c', 'request_id': 'req-c109f707-0198-402e-adb0-cb5b4faaceac'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:00.773599 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4b184-ab5a-42cf-9c5a-8f87d21c0c61 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:00.786965 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c109f707-0198-402e-adb0-cb5b4faaceac admin admin] The token cache is being invalidate because project 2e0c164b1bca495ead041f096ccf073c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:20:00.787374 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c109f707-0198-402e-adb0-cb5b4faaceac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:00.789756 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c109f707-0198-402e-adb0-cb5b4faaceac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:00.792474 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1012/2019] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:20:00 2022] DELETE /identity/v3/projects/2e0c164b1bca495ead041f096ccf073c => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:20:00.796818 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c109f707-0198-402e-adb0-cb5b4faaceac admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:00.801077 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c109f707-0198-402e-adb0-cb5b4faaceac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:00.915444 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9538b799-bcd7-45cf-8150-5dbfb7074230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:00.919245 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9538b799-bcd7-45cf-8150-5dbfb7074230 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:00.919914 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9538b799-bcd7-45cf-8150-5dbfb7074230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:00.920736 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9538b799-bcd7-45cf-8150-5dbfb7074230 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:00.922208 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9538b799-bcd7-45cf-8150-5dbfb7074230 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:00.924217 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9538b799-bcd7-45cf-8150-5dbfb7074230 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:00.942994 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1008/2020] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:20:00 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:20:00.945427 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9538b799-bcd7-45cf-8150-5dbfb7074230 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:00.947886 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-9538b799-bcd7-45cf-8150-5dbfb7074230 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:00.949260 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c109f707-0198-402e-adb0-cb5b4faaceac admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:00.958093 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07681e7-f31b-4e9a-80ec-a57e5075dc7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:00.960465 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07681e7-f31b-4e9a-80ec-a57e5075dc7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:00.960855 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07681e7-f31b-4e9a-80ec-a57e5075dc7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:00.961171 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07681e7-f31b-4e9a-80ec-a57e5075dc7d admin admin] PATH_INFO: `/v3/projects/41ed734c32544b988ee1df0486a10d37` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:00.966804 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07681e7-f31b-4e9a-80ec-a57e5075dc7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=41ed734c32544b988ee1df0486a10d37)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:00.971629 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9538b799-bcd7-45cf-8150-5dbfb7074230 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:00.971876 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07681e7-f31b-4e9a-80ec-a57e5075dc7d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:00.978963 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3471585f-11ac-4ca0-a76d-939d8616d357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:00.980138 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471585f-11ac-4ca0-a76d-939d8616d357 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:00.980506 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471585f-11ac-4ca0-a76d-939d8616d357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:00.980863 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471585f-11ac-4ca0-a76d-939d8616d357 admin admin] PATH_INFO: `/v3/users/72c239db8e164300809154ee03dd77da` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:00.982258 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3471585f-11ac-4ca0-a76d-939d8616d357 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72c239db8e164300809154ee03dd77da)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:00.984666 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3471585f-11ac-4ca0-a76d-939d8616d357 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:01.033392 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:01.080387 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:01.087747 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-3471585f-11ac-4ca0-a76d-939d8616d357 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72c239db8e164300809154ee03dd77da'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:01.095969 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f07681e7-f31b-4e9a-80ec-a57e5075dc7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41ed734c32544b988ee1df0486a10d37', 'request_id': 'req-f07681e7-f31b-4e9a-80ec-a57e5075dc7d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:01.107927 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-3471585f-11ac-4ca0-a76d-939d8616d357 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72c239db8e164300809154ee03dd77da'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:01.108714 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-3471585f-11ac-4ca0-a76d-939d8616d357 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72c239db8e164300809154ee03dd77da'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:01.108863 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f07681e7-f31b-4e9a-80ec-a57e5075dc7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41ed734c32544b988ee1df0486a10d37', 'request_id': 'req-f07681e7-f31b-4e9a-80ec-a57e5075dc7d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:01.120036 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f07681e7-f31b-4e9a-80ec-a57e5075dc7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41ed734c32544b988ee1df0486a10d37', 'request_id': 'req-f07681e7-f31b-4e9a-80ec-a57e5075dc7d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:01.124620 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-3471585f-11ac-4ca0-a76d-939d8616d357 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72c239db8e164300809154ee03dd77da'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:01.137211 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f07681e7-f31b-4e9a-80ec-a57e5075dc7d admin admin] The token cache is being invalidate because project 41ed734c32544b988ee1df0486a10d37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:20:01.140683 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f07681e7-f31b-4e9a-80ec-a57e5075dc7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:01.141644 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f07681e7-f31b-4e9a-80ec-a57e5075dc7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:01.141996 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-3471585f-11ac-4ca0-a76d-939d8616d357 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72c239db8e164300809154ee03dd77da'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:01.148578 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1013/2021] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:20:00 2022] DELETE /identity/v3/projects/41ed734c32544b988ee1df0486a10d37 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:20:01.149750 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07681e7-f31b-4e9a-80ec-a57e5075dc7d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:01.151359 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f07681e7-f31b-4e9a-80ec-a57e5075dc7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:01.166733 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1009/2022] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:20:00 2022] DELETE /identity/v3/users/72c239db8e164300809154ee03dd77da => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:20:01.175030 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3193c1a7-a354-43ae-b221-bccd579f86d0 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:01.175490 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3193c1a7-a354-43ae-b221-bccd579f86d0 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:01.175781 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3193c1a7-a354-43ae-b221-bccd579f86d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:01.220095 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-3193c1a7-a354-43ae-b221-bccd579f86d0 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:01.221525 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-3193c1a7-a354-43ae-b221-bccd579f86d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:01.245271 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07681e7-f31b-4e9a-80ec-a57e5075dc7d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:01.360909 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96b1882-7bd0-4efd-ae2d-9a303e932899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:01.362901 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b1882-7bd0-4efd-ae2d-9a303e932899 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:01.363431 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b1882-7bd0-4efd-ae2d-9a303e932899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:01.363714 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b1882-7bd0-4efd-ae2d-9a303e932899 admin admin] PATH_INFO: `/v3/projects/d2b544835c5945dda00145d930dcedd3/users/35f3dd6ab14c42f6928f09fa24c1ebe3/roles/f36e053b442540f08041ac7878760f65` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:01.407181 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96b1882-7bd0-4efd-ae2d-9a303e932899 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2b544835c5945dda00145d930dcedd3, user_id=35f3dd6ab14c42f6928f09fa24c1ebe3, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:01.408931 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1010/2023] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:20:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:20:01.411202 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96b1882-7bd0-4efd-ae2d-9a303e932899 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:01.417972 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d371e8a-a66f-431d-bcb4-b39fab256afb None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:01.418267 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d371e8a-a66f-431d-bcb4-b39fab256afb None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:01.418560 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d371e8a-a66f-431d-bcb4-b39fab256afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:01.454984 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-8d371e8a-a66f-431d-bcb4-b39fab256afb None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:01.458405 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-8d371e8a-a66f-431d-bcb4-b39fab256afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:01.488211 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1014/2024] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:20:01 2022] PUT /identity/v3/projects/d2b544835c5945dda00145d930dcedd3/users/35f3dd6ab14c42f6928f09fa24c1ebe3/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:20:01.490646 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96b1882-7bd0-4efd-ae2d-9a303e932899 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:01.606652 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:01.638394 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1011/2025] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:20:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:20:01.640489 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d371e8a-a66f-431d-bcb4-b39fab256afb None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:01.650298 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96b1882-7bd0-4efd-ae2d-9a303e932899 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:01.657280 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db134b8d-fb60-4a70-a97c-d1ab3c742f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:01.658059 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db134b8d-fb60-4a70-a97c-d1ab3c742f8d admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:01.658340 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db134b8d-fb60-4a70-a97c-d1ab3c742f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:01.658614 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db134b8d-fb60-4a70-a97c-d1ab3c742f8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:01.659884 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db134b8d-fb60-4a70-a97c-d1ab3c742f8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:01.662721 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db134b8d-fb60-4a70-a97c-d1ab3c742f8d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:01.665490 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d371e8a-a66f-431d-bcb4-b39fab256afb None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:01.677320 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac8b49e-c772-4fe8-9700-9ec5af4726cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:01.679895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac8b49e-c772-4fe8-9700-9ec5af4726cf admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:01.680485 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac8b49e-c772-4fe8-9700-9ec5af4726cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:01.680988 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac8b49e-c772-4fe8-9700-9ec5af4726cf admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:01.682385 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac8b49e-c772-4fe8-9700-9ec5af4726cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:20:01.683135 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac8b49e-c772-4fe8-9700-9ec5af4726cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:01.685687 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1015/2026] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:20:01 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:20:01.686095 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac8b49e-c772-4fe8-9700-9ec5af4726cf admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:01.686627 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db134b8d-fb60-4a70-a97c-d1ab3c742f8d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:01.689940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-db134b8d-fb60-4a70-a97c-d1ab3c742f8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:01.705280 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db134b8d-fb60-4a70-a97c-d1ab3c742f8d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:01.707744 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1012/2027] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:20:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:20:01.708780 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac8b49e-c772-4fe8-9700-9ec5af4726cf admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:01.715114 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf44f8d5-655e-42a8-81a9-2d9e40dc5cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:01.716669 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44f8d5-655e-42a8-81a9-2d9e40dc5cef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:01.717723 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44f8d5-655e-42a8-81a9-2d9e40dc5cef admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:01.718160 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44f8d5-655e-42a8-81a9-2d9e40dc5cef admin admin] PATH_INFO: `/v3/projects/d4e74710ed0547e886f0bd869948807a` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:01.719897 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf44f8d5-655e-42a8-81a9-2d9e40dc5cef admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4e74710ed0547e886f0bd869948807a)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:01.721889 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac8b49e-c772-4fe8-9700-9ec5af4726cf admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:01.722465 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf44f8d5-655e-42a8-81a9-2d9e40dc5cef admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:01.731168 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ae1092-1caa-45fd-b01d-8bb678e49f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:01.735680 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae1092-1caa-45fd-b01d-8bb678e49f6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:01.735680 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae1092-1caa-45fd-b01d-8bb678e49f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:01.735680 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae1092-1caa-45fd-b01d-8bb678e49f6f admin admin] PATH_INFO: `/v3/projects/d2b544835c5945dda00145d930dcedd3/users/35f3dd6ab14c42f6928f09fa24c1ebe3/roles/f36e053b442540f08041ac7878760f65` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:01.739682 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ae1092-1caa-45fd-b01d-8bb678e49f6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2b544835c5945dda00145d930dcedd3, user_id=35f3dd6ab14c42f6928f09fa24c1ebe3, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:01.746718 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ae1092-1caa-45fd-b01d-8bb678e49f6f admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:01.764132 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1013/2028] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:20:01 2022] PUT /identity/v3/projects/d2b544835c5945dda00145d930dcedd3/users/35f3dd6ab14c42f6928f09fa24c1ebe3/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:20:01.770812 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ae1092-1caa-45fd-b01d-8bb678e49f6f admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:01.836128 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bf44f8d5-655e-42a8-81a9-2d9e40dc5cef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4e74710ed0547e886f0bd869948807a', 'request_id': 'req-bf44f8d5-655e-42a8-81a9-2d9e40dc5cef'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:01.850996 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bf44f8d5-655e-42a8-81a9-2d9e40dc5cef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4e74710ed0547e886f0bd869948807a', 'request_id': 'req-bf44f8d5-655e-42a8-81a9-2d9e40dc5cef'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:01.853703 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:01.871667 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bf44f8d5-655e-42a8-81a9-2d9e40dc5cef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4e74710ed0547e886f0bd869948807a', 'request_id': 'req-bf44f8d5-655e-42a8-81a9-2d9e40dc5cef'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:01.894339 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bf44f8d5-655e-42a8-81a9-2d9e40dc5cef admin admin] The token cache is being invalidate because project d4e74710ed0547e886f0bd869948807a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:20:01.894763 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bf44f8d5-655e-42a8-81a9-2d9e40dc5cef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:01.895559 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bf44f8d5-655e-42a8-81a9-2d9e40dc5cef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:01.903803 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1016/2029] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:20:01 2022] DELETE /identity/v3/projects/d4e74710ed0547e886f0bd869948807a => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:20:01.903803 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca77d5fd-bc82-477f-94cf-21ab3a50992d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:01.903803 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca77d5fd-bc82-477f-94cf-21ab3a50992d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:01.903803 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca77d5fd-bc82-477f-94cf-21ab3a50992d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:01.910509 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ae1092-1caa-45fd-b01d-8bb678e49f6f admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:01.949953 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ca77d5fd-bc82-477f-94cf-21ab3a50992d None None] MFA Rules not processed for user `35f3dd6ab14c42f6928f09fa24c1ebe3`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:01.951821 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ca77d5fd-bc82-477f-94cf-21ab3a50992d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:02.006351 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6ea45b-4e60-4dc0-8282-5ba6d7e45e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:02.007955 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ea45b-4e60-4dc0-8282-5ba6d7e45e33 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:02.008216 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ea45b-4e60-4dc0-8282-5ba6d7e45e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:02.008484 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ea45b-4e60-4dc0-8282-5ba6d7e45e33 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:02.010288 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6ea45b-4e60-4dc0-8282-5ba6d7e45e33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:02.012776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6ea45b-4e60-4dc0-8282-5ba6d7e45e33 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:02.047102 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1014/2030] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:20:01 2022] POST /identity/v3/projects => generated 377 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:20:02.048535 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e892c-e580-42a2-92f4-2b00da773c40 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:02.048981 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e892c-e580-42a2-92f4-2b00da773c40 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:02.049302 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e892c-e580-42a2-92f4-2b00da773c40 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:02.051048 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1015/2031] 10.222.0.71 () {54 vars in 921 bytes} [Mon Jul 4 09:20:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:20:02.052198 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a62397-68de-4d8d-b0bb-281bf18cbb3f None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:02.052470 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a62397-68de-4d8d-b0bb-281bf18cbb3f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:02.052756 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a62397-68de-4d8d-b0bb-281bf18cbb3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:02.069850 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1017/2032] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:20:01 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:20:02.071836 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca77d5fd-bc82-477f-94cf-21ab3a50992d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:02.078549 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ca77d5fd-bc82-477f-94cf-21ab3a50992d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:02.090217 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-c9a62397-68de-4d8d-b0bb-281bf18cbb3f None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:02.091698 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c9a62397-68de-4d8d-b0bb-281bf18cbb3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:02.107029 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca77d5fd-bc82-477f-94cf-21ab3a50992d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:02.181365 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:02.192611 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1016/2033] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:20:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:20:02.202569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655863ec-8dca-46a7-9cc3-e32d2ce148ab None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:02.205751 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655863ec-8dca-46a7-9cc3-e32d2ce148ab None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:02.206503 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655863ec-8dca-46a7-9cc3-e32d2ce148ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:02.209236 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b9d7c6-914e-4bfb-ac88-ad607444c06c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:02.210918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b9d7c6-914e-4bfb-ac88-ad607444c06c admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:02.211408 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b9d7c6-914e-4bfb-ac88-ad607444c06c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:02.211889 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b9d7c6-914e-4bfb-ac88-ad607444c06c admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:02.213421 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b9d7c6-914e-4bfb-ac88-ad607444c06c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:02.216343 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b9d7c6-914e-4bfb-ac88-ad607444c06c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:02.248890 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-655863ec-8dca-46a7-9cc3-e32d2ce148ab None None] MFA Rules not processed for user `15039da209ec445d93d49e2bd6d854eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:02.251494 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-655863ec-8dca-46a7-9cc3-e32d2ce148ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:02.252554 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1018/2034] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:20:02 2022] POST /identity/v3/users => generated 289 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:20:02.253526 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b9d7c6-914e-4bfb-ac88-ad607444c06c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:02.255483 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-85b9d7c6-914e-4bfb-ac88-ad607444c06c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:02.351421 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b9d7c6-914e-4bfb-ac88-ad607444c06c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:02.359644 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29670e84-fe0a-4038-8d6c-fbeb88c8f056 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:02.360588 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29670e84-fe0a-4038-8d6c-fbeb88c8f056 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:02.361218 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29670e84-fe0a-4038-8d6c-fbeb88c8f056 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:02.362574 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29670e84-fe0a-4038-8d6c-fbeb88c8f056 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:02.364781 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-29670e84-fe0a-4038-8d6c-fbeb88c8f056 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:02.374944 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1017/2035] 10.222.0.71 () {58 vars in 1018 bytes} [Mon Jul 4 09:20:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:20:02.374944 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb67158-841a-4f9e-93ee-6fe67ca4e52e None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:02.374944 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb67158-841a-4f9e-93ee-6fe67ca4e52e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:02.374944 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb67158-841a-4f9e-93ee-6fe67ca4e52e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:02.395971 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:02.408194 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-1eb67158-841a-4f9e-93ee-6fe67ca4e52e None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:02.410052 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-1eb67158-841a-4f9e-93ee-6fe67ca4e52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:02.421187 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1018/2036] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:20:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:20:02.422223 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb67158-841a-4f9e-93ee-6fe67ca4e52e None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:02.432926 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb67158-841a-4f9e-93ee-6fe67ca4e52e None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:02.438602 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e717d57-061c-4512-88de-ae741afe1768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:02.439457 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e717d57-061c-4512-88de-ae741afe1768 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:02.439873 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e717d57-061c-4512-88de-ae741afe1768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:02.442158 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e717d57-061c-4512-88de-ae741afe1768 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:02.443010 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e717d57-061c-4512-88de-ae741afe1768 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:02.445642 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e717d57-061c-4512-88de-ae741afe1768 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:02.465358 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1019/2037] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:20:02 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:20:02.467247 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e717d57-061c-4512-88de-ae741afe1768 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:02.468829 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29670e84-fe0a-4038-8d6c-fbeb88c8f056 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:02.469503 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-6e717d57-061c-4512-88de-ae741afe1768 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:02.470486 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29670e84-fe0a-4038-8d6c-fbeb88c8f056 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:02.485017 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1019/2038] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:20:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:20:02.488469 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e717d57-061c-4512-88de-ae741afe1768 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:02.496089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658dc02e-69b2-43ac-ba5c-362649f572ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15039da209ec445d93d49e2bd6d854eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:02.496941 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658dc02e-69b2-43ac-ba5c-362649f572ca service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:02.497427 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658dc02e-69b2-43ac-ba5c-362649f572ca service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:02.497864 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658dc02e-69b2-43ac-ba5c-362649f572ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:02.499441 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29670e84-fe0a-4038-8d6c-fbeb88c8f056 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:02.509687 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658dc02e-69b2-43ac-ba5c-362649f572ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:02.511956 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658dc02e-69b2-43ac-ba5c-362649f572ca service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:02.525232 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1020/2039] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:20:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:20:02.526392 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29670e84-fe0a-4038-8d6c-fbeb88c8f056 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:02.526901 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658dc02e-69b2-43ac-ba5c-362649f572ca service nova] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:02.533003 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8e6df-e293-4f33-84a6-d52e2baf2db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:02.534487 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8e6df-e293-4f33-84a6-d52e2baf2db6 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:02.534880 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8e6df-e293-4f33-84a6-d52e2baf2db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:02.535267 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8e6df-e293-4f33-84a6-d52e2baf2db6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:02.536255 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c8e6df-e293-4f33-84a6-d52e2baf2db6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:20:02.536671 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c8e6df-e293-4f33-84a6-d52e2baf2db6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:02.542163 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c8e6df-e293-4f33-84a6-d52e2baf2db6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:02.549329 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658dc02e-69b2-43ac-ba5c-362649f572ca service nova] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:02.562577 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1020/2040] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:20:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:20:02.563168 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cd35b9-26e6-4132-b0b2-9a2f31edeb03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:02.563863 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd35b9-26e6-4132-b0b2-9a2f31edeb03 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:02.564147 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd35b9-26e6-4132-b0b2-9a2f31edeb03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:02.570118 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8e6df-e293-4f33-84a6-d52e2baf2db6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:02.574873 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd35b9-26e6-4132-b0b2-9a2f31edeb03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:02.575723 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cd35b9-26e6-4132-b0b2-9a2f31edeb03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:02.577384 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cd35b9-26e6-4132-b0b2-9a2f31edeb03 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:02.592285 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8e6df-e293-4f33-84a6-d52e2baf2db6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:02.601297 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1021/2041] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:20:02 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:20:02.602211 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cd35b9-26e6-4132-b0b2-9a2f31edeb03 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:02.602918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f554beb-9ff5-4aab-a197-cfda4521ad28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:02.603788 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f554beb-9ff5-4aab-a197-cfda4521ad28 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:02.604896 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f554beb-9ff5-4aab-a197-cfda4521ad28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:02.605317 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f554beb-9ff5-4aab-a197-cfda4521ad28 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:02.606288 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f554beb-9ff5-4aab-a197-cfda4521ad28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:02.614930 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f554beb-9ff5-4aab-a197-cfda4521ad28 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:02.626862 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cd35b9-26e6-4132-b0b2-9a2f31edeb03 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:02.660416 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca13cc-26d8-48d9-9e22-a22dcf9793b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:02.661301 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1021/2042] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:20:02 2022] POST /identity/v3/projects => generated 383 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:20:02.661455 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca13cc-26d8-48d9-9e22-a22dcf9793b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:02.661669 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca13cc-26d8-48d9-9e22-a22dcf9793b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:02.662001 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca13cc-26d8-48d9-9e22-a22dcf9793b8 admin admin] PATH_INFO: `/v3/projects/58c3e99e1a884a59966a4c18d934bbac/users/6db1042af2724042aa96119b5ce3147f/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:02.666662 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f554beb-9ff5-4aab-a197-cfda4521ad28 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:02.710506 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dca13cc-26d8-48d9-9e22-a22dcf9793b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58c3e99e1a884a59966a4c18d934bbac, user_id=6db1042af2724042aa96119b5ce3147f, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:02.713167 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dca13cc-26d8-48d9-9e22-a22dcf9793b8 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:02.746071 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:02.750783 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1022/2043] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:20:02 2022] PUT /identity/v3/projects/58c3e99e1a884a59966a4c18d934bbac/users/6db1042af2724042aa96119b5ce3147f/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:20:02.765350 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f554beb-9ff5-4aab-a197-cfda4521ad28 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:02.768077 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45560ed-3f07-4b0d-9c49-f87ca9f74405 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:02.768362 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45560ed-3f07-4b0d-9c49-f87ca9f74405 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:02.768641 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45560ed-3f07-4b0d-9c49-f87ca9f74405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:02.835368 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-c45560ed-3f07-4b0d-9c49-f87ca9f74405 None None] MFA Rules not processed for user `6db1042af2724042aa96119b5ce3147f`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:02.837072 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c45560ed-3f07-4b0d-9c49-f87ca9f74405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:02.885727 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d3784-2d6a-4a78-9667-e1c6a152a3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:02.886410 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d3784-2d6a-4a78-9667-e1c6a152a3e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:02.889486 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d3784-2d6a-4a78-9667-e1c6a152a3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:02.890456 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d3784-2d6a-4a78-9667-e1c6a152a3e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:02.891517 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75d3784-2d6a-4a78-9667-e1c6a152a3e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:02.895659 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75d3784-2d6a-4a78-9667-e1c6a152a3e3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:02.948112 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1023/2044] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:20:02 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:20:02.949994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1022/2045] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:20:02 2022] POST /identity/v3/users => generated 292 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:20:02.954833 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45560ed-3f07-4b0d-9c49-f87ca9f74405 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:02.974835 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45560ed-3f07-4b0d-9c49-f87ca9f74405 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:02.984184 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb2338-542b-448f-8a83-c1ea4ba259db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:02.985288 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb2338-542b-448f-8a83-c1ea4ba259db admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:02.985769 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb2338-542b-448f-8a83-c1ea4ba259db admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:02.986226 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb2338-542b-448f-8a83-c1ea4ba259db admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:02.987245 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fb2338-542b-448f-8a83-c1ea4ba259db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:02.990226 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fb2338-542b-448f-8a83-c1ea4ba259db admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:03.007942 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1024/2046] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:20:02 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:20:03.008115 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d3784-2d6a-4a78-9667-e1c6a152a3e3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:03.023719 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d3784-2d6a-4a78-9667-e1c6a152a3e3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:03.037142 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7fb482-25bb-46e2-9691-deea227008e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:03.037951 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7fb482-25bb-46e2-9691-deea227008e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:03.039115 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7fb482-25bb-46e2-9691-deea227008e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:03.039499 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7fb482-25bb-46e2-9691-deea227008e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:03.040316 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7fb482-25bb-46e2-9691-deea227008e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:03.041964 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7fb482-25bb-46e2-9691-deea227008e7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:03.054026 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1023/2047] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:20:03 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:20:03.059116 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb2338-542b-448f-8a83-c1ea4ba259db admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:03.074734 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb2338-542b-448f-8a83-c1ea4ba259db admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:03.080272 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daca5848-753f-4630-9e18-78ae77c318d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:03.081254 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca5848-753f-4630-9e18-78ae77c318d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:03.081553 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca5848-753f-4630-9e18-78ae77c318d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:03.082028 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca5848-753f-4630-9e18-78ae77c318d9 admin admin] PATH_INFO: `/v3/projects/e71d17cf5e3641c9b588f7e6964e6840/users/57786d6b5a924a3cab1755421d1095c6/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:03.108757 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daca5848-753f-4630-9e18-78ae77c318d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e71d17cf5e3641c9b588f7e6964e6840, user_id=57786d6b5a924a3cab1755421d1095c6, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:03.114372 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daca5848-753f-4630-9e18-78ae77c318d9 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:03.135281 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1025/2048] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:20:03 2022] PUT /identity/v3/projects/e71d17cf5e3641c9b588f7e6964e6840/users/57786d6b5a924a3cab1755421d1095c6/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:20:03.154036 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252bff74-4bd0-426e-a395-2df836d6c22c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:03.154595 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252bff74-4bd0-426e-a395-2df836d6c22c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:03.155069 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252bff74-4bd0-426e-a395-2df836d6c22c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:03.216287 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-252bff74-4bd0-426e-a395-2df836d6c22c None None] MFA Rules not processed for user `57786d6b5a924a3cab1755421d1095c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:03.217568 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-252bff74-4bd0-426e-a395-2df836d6c22c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:03.321278 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daca5848-753f-4630-9e18-78ae77c318d9 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:03.348243 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1024/2049] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:20:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:20:03.354029 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252bff74-4bd0-426e-a395-2df836d6c22c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:03.463695 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daca5848-753f-4630-9e18-78ae77c318d9 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:03.470487 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc7f7e1-f2f5-457f-ae3e-483c81f16170 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:03.471291 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7f7e1-f2f5-457f-ae3e-483c81f16170 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:03.471586 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7f7e1-f2f5-457f-ae3e-483c81f16170 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:03.471868 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7f7e1-f2f5-457f-ae3e-483c81f16170 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:03.478992 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:03.483374 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252bff74-4bd0-426e-a395-2df836d6c22c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:03.491973 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb30863-3b9b-4be1-bd08-f49767f99a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:03.493201 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb30863-3b9b-4be1-bd08-f49767f99a13 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:03.493496 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb30863-3b9b-4be1-bd08-f49767f99a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:03.493821 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb30863-3b9b-4be1-bd08-f49767f99a13 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:03.495021 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb30863-3b9b-4be1-bd08-f49767f99a13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:03.496981 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb30863-3b9b-4be1-bd08-f49767f99a13 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:03.528000 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:03.563119 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1025/2050] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:20:03 2022] POST /identity/v3/projects => generated 383 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:20:03.563936 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc7f7e1-f2f5-457f-ae3e-483c81f16170 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:03.565706 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc7f7e1-f2f5-457f-ae3e-483c81f16170 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:03.566042 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb30863-3b9b-4be1-bd08-f49767f99a13 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:03.576970 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1026/2051] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:20:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:20:03.675634 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb30863-3b9b-4be1-bd08-f49767f99a13 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:03.686137 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819d0529-ffc7-4cc0-9ce3-97db64ab1d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:03.688724 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d0529-ffc7-4cc0-9ce3-97db64ab1d93 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:03.689168 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d0529-ffc7-4cc0-9ce3-97db64ab1d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:03.689597 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d0529-ffc7-4cc0-9ce3-97db64ab1d93 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:03.690456 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819d0529-ffc7-4cc0-9ce3-97db64ab1d93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:03.692781 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819d0529-ffc7-4cc0-9ce3-97db64ab1d93 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:03.718859 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1026/2052] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:20:03 2022] POST /identity/v3/users => generated 298 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:20:03.722870 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc7f7e1-f2f5-457f-ae3e-483c81f16170 service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:03.738676 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc7f7e1-f2f5-457f-ae3e-483c81f16170 service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:03.745060 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5451c114-2698-407c-bbc8-f0a9a4cb5fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:03.745885 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5451c114-2698-407c-bbc8-f0a9a4cb5fd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:03.746085 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5451c114-2698-407c-bbc8-f0a9a4cb5fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:03.748556 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5451c114-2698-407c-bbc8-f0a9a4cb5fd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:03.748556 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5451c114-2698-407c-bbc8-f0a9a4cb5fd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:03.749235 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5451c114-2698-407c-bbc8-f0a9a4cb5fd5 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:03.764588 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1027/2053] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:20:03 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:20:03.768971 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819d0529-ffc7-4cc0-9ce3-97db64ab1d93 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:03.781066 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819d0529-ffc7-4cc0-9ce3-97db64ab1d93 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:03.786991 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9a89ee-9901-4bcc-948b-4ae211658e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:03.788676 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9a89ee-9901-4bcc-948b-4ae211658e59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:03.797671 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9a89ee-9901-4bcc-948b-4ae211658e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:03.798114 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9a89ee-9901-4bcc-948b-4ae211658e59 admin admin] PATH_INFO: `/v3/projects/b030c2ed96da4869875c35e43964dc10/users/eab6da6450c54419a334c451862c0b96/roles/f36e053b442540f08041ac7878760f65` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:03.823703 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9a89ee-9901-4bcc-948b-4ae211658e59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b030c2ed96da4869875c35e43964dc10, user_id=eab6da6450c54419a334c451862c0b96, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:03.826512 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9a89ee-9901-4bcc-948b-4ae211658e59 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:03.840900 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1027/2054] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:20:03 2022] PUT /identity/v3/projects/b030c2ed96da4869875c35e43964dc10/users/eab6da6450c54419a334c451862c0b96/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:20:03.845448 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5451c114-2698-407c-bbc8-f0a9a4cb5fd5 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:03.859109 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5451c114-2698-407c-bbc8-f0a9a4cb5fd5 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:03.952021 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4315a8-55f6-4410-b570-3162aa89f13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:03.952737 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4315a8-55f6-4410-b570-3162aa89f13a admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:03.953031 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4315a8-55f6-4410-b570-3162aa89f13a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:03.953366 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4315a8-55f6-4410-b570-3162aa89f13a admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:03.954662 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4315a8-55f6-4410-b570-3162aa89f13a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:03.955811 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4315a8-55f6-4410-b570-3162aa89f13a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:03.969366 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1028/2055] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:20:03 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:20:03.973581 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9a89ee-9901-4bcc-948b-4ae211658e59 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:03.985368 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9a89ee-9901-4bcc-948b-4ae211658e59 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:03.990180 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f501ccfb-91cd-491e-9ab0-a342b8f3cd2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:03.991895 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f501ccfb-91cd-491e-9ab0-a342b8f3cd2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:03.992250 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f501ccfb-91cd-491e-9ab0-a342b8f3cd2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:03.992657 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f501ccfb-91cd-491e-9ab0-a342b8f3cd2f admin admin] PATH_INFO: `/v3/projects/b030c2ed96da4869875c35e43964dc10/users/eab6da6450c54419a334c451862c0b96/roles/f36e053b442540f08041ac7878760f65` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:03.994861 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f501ccfb-91cd-491e-9ab0-a342b8f3cd2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b030c2ed96da4869875c35e43964dc10, user_id=eab6da6450c54419a334c451862c0b96, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:03.997714 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f501ccfb-91cd-491e-9ab0-a342b8f3cd2f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:04.008373 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1028/2056] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:20:03 2022] PUT /identity/v3/projects/b030c2ed96da4869875c35e43964dc10/users/eab6da6450c54419a334c451862c0b96/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:20:04.018792 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c25c0-c83c-492a-8c22-2e4545f36d71 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:04.019031 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c25c0-c83c-492a-8c22-2e4545f36d71 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:04.019349 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c25c0-c83c-492a-8c22-2e4545f36d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:04.046051 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-2a5c25c0-c83c-492a-8c22-2e4545f36d71 None None] MFA Rules not processed for user `eab6da6450c54419a334c451862c0b96`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:04.047256 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-2a5c25c0-c83c-492a-8c22-2e4545f36d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:04.115299 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1029/2057] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:20:04 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:20:04.126987 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f501ccfb-91cd-491e-9ab0-a342b8f3cd2f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:04.186813 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f501ccfb-91cd-491e-9ab0-a342b8f3cd2f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:04.190511 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b45ce85-b8b3-4b1c-a862-ab28483625b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:04.191161 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b45ce85-b8b3-4b1c-a862-ab28483625b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:04.191448 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b45ce85-b8b3-4b1c-a862-ab28483625b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:04.191730 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b45ce85-b8b3-4b1c-a862-ab28483625b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:04.200759 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b45ce85-b8b3-4b1c-a862-ab28483625b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:04.202231 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b45ce85-b8b3-4b1c-a862-ab28483625b7 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:04.212187 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1029/2058] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:20:04 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:20:04.318548 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5c25c0-c83c-492a-8c22-2e4545f36d71 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:04.328780 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5c25c0-c83c-492a-8c22-2e4545f36d71 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:04.332399 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aeed96-7805-41a1-a19d-748d33f36f75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:04.333060 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aeed96-7805-41a1-a19d-748d33f36f75 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:04.333364 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aeed96-7805-41a1-a19d-748d33f36f75 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:04.333647 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aeed96-7805-41a1-a19d-748d33f36f75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:04.345949 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:04.394692 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aeed96-7805-41a1-a19d-748d33f36f75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:04.396138 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aeed96-7805-41a1-a19d-748d33f36f75 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:04.405467 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1030/2059] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:20:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:20:09.458558 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b45ce85-b8b3-4b1c-a862-ab28483625b7 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:09.460157 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4b45ce85-b8b3-4b1c-a862-ab28483625b7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:09.476963 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b45ce85-b8b3-4b1c-a862-ab28483625b7 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:09.485750 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b391a3f-b4f0-4f61-ab8d-60dd3e8607a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:09.486456 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b391a3f-b4f0-4f61-ab8d-60dd3e8607a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:09.486791 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b391a3f-b4f0-4f61-ab8d-60dd3e8607a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:09.487086 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b391a3f-b4f0-4f61-ab8d-60dd3e8607a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:09.544886 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:09.564118 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b391a3f-b4f0-4f61-ab8d-60dd3e8607a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:09.565688 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b391a3f-b4f0-4f61-ab8d-60dd3e8607a4 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:09.576720 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1030/2060] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:20:09 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:20:10.131550 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aeed96-7805-41a1-a19d-748d33f36f75 service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:10.206642 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aeed96-7805-41a1-a19d-748d33f36f75 service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:10.212018 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2e7936-6a8c-48aa-afad-537fbe7a4436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:10.213190 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2e7936-6a8c-48aa-afad-537fbe7a4436 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:10.213471 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2e7936-6a8c-48aa-afad-537fbe7a4436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:10.213757 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2e7936-6a8c-48aa-afad-537fbe7a4436 admin admin] PATH_INFO: `/v3/users/57786d6b5a924a3cab1755421d1095c6` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:10.215385 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2e7936-6a8c-48aa-afad-537fbe7a4436 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57786d6b5a924a3cab1755421d1095c6)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:10.217353 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2e7936-6a8c-48aa-afad-537fbe7a4436 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:10.264671 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-9f2e7936-6a8c-48aa-afad-537fbe7a4436 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57786d6b5a924a3cab1755421d1095c6'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:10.272243 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-9f2e7936-6a8c-48aa-afad-537fbe7a4436 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57786d6b5a924a3cab1755421d1095c6'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:10.272953 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-9f2e7936-6a8c-48aa-afad-537fbe7a4436 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57786d6b5a924a3cab1755421d1095c6'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:10.281198 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-9f2e7936-6a8c-48aa-afad-537fbe7a4436 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57786d6b5a924a3cab1755421d1095c6'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:10.294426 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-9f2e7936-6a8c-48aa-afad-537fbe7a4436 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57786d6b5a924a3cab1755421d1095c6'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:10.300742 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1031/2061] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:20:10 2022] DELETE /identity/v3/users/57786d6b5a924a3cab1755421d1095c6 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:20:10.305079 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb848a-87b3-4d12-b31a-0658dc089d7f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:10.306045 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb848a-87b3-4d12-b31a-0658dc089d7f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:10.306401 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb848a-87b3-4d12-b31a-0658dc089d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:10.325411 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-18fb848a-87b3-4d12-b31a-0658dc089d7f None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:10.326757 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-18fb848a-87b3-4d12-b31a-0658dc089d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:10.387967 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1031/2062] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:20:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:20:10.400281 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2e7936-6a8c-48aa-afad-537fbe7a4436 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:10.401754 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-9f2e7936-6a8c-48aa-afad-537fbe7a4436 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:10.439085 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:10.465479 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2e7936-6a8c-48aa-afad-537fbe7a4436 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:10.469564 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6eda56-2057-4a23-a191-c1713ac10e86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:10.470059 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6eda56-2057-4a23-a191-c1713ac10e86 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:10.470868 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6eda56-2057-4a23-a191-c1713ac10e86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:10.470868 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6eda56-2057-4a23-a191-c1713ac10e86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:10.483259 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6eda56-2057-4a23-a191-c1713ac10e86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:10.484963 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6eda56-2057-4a23-a191-c1713ac10e86 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:10.499150 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1032/2063] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:20:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:20:10.943365 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fb848a-87b3-4d12-b31a-0658dc089d7f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:10.945025 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-18fb848a-87b3-4d12-b31a-0658dc089d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:10.959221 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fb848a-87b3-4d12-b31a-0658dc089d7f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:10.965589 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d32e71-8726-44a5-908e-9d1b64611a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:10.966534 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d32e71-8726-44a5-908e-9d1b64611a0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:10.966856 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d32e71-8726-44a5-908e-9d1b64611a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:10.967166 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d32e71-8726-44a5-908e-9d1b64611a0c admin admin] PATH_INFO: `/v3/projects/e71d17cf5e3641c9b588f7e6964e6840` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:10.968549 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d32e71-8726-44a5-908e-9d1b64611a0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e71d17cf5e3641c9b588f7e6964e6840)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:10.970511 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d32e71-8726-44a5-908e-9d1b64611a0c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:11.023197 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-74d32e71-8726-44a5-908e-9d1b64611a0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e71d17cf5e3641c9b588f7e6964e6840', 'request_id': 'req-74d32e71-8726-44a5-908e-9d1b64611a0c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:11.029461 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-74d32e71-8726-44a5-908e-9d1b64611a0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e71d17cf5e3641c9b588f7e6964e6840', 'request_id': 'req-74d32e71-8726-44a5-908e-9d1b64611a0c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:11.036206 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-74d32e71-8726-44a5-908e-9d1b64611a0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e71d17cf5e3641c9b588f7e6964e6840', 'request_id': 'req-74d32e71-8726-44a5-908e-9d1b64611a0c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:11.050784 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-74d32e71-8726-44a5-908e-9d1b64611a0c admin admin] The token cache is being invalidate because project e71d17cf5e3641c9b588f7e6964e6840 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:20:11.051115 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-74d32e71-8726-44a5-908e-9d1b64611a0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:11.051748 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-74d32e71-8726-44a5-908e-9d1b64611a0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:11.053370 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1032/2064] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:20:10 2022] DELETE /identity/v3/projects/e71d17cf5e3641c9b588f7e6964e6840 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:20:11.058013 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6eda56-2057-4a23-a191-c1713ac10e86 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:11.068417 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6eda56-2057-4a23-a191-c1713ac10e86 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:11.128178 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da73d03d-c54e-4a8d-bcf5-0e4e7465bb99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:11.129093 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da73d03d-c54e-4a8d-bcf5-0e4e7465bb99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:11.129394 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da73d03d-c54e-4a8d-bcf5-0e4e7465bb99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:11.129669 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da73d03d-c54e-4a8d-bcf5-0e4e7465bb99 admin admin] PATH_INFO: `/v3/users/eab6da6450c54419a334c451862c0b96` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:11.130896 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da73d03d-c54e-4a8d-bcf5-0e4e7465bb99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eab6da6450c54419a334c451862c0b96)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:11.132843 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da73d03d-c54e-4a8d-bcf5-0e4e7465bb99 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:11.184405 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-da73d03d-c54e-4a8d-bcf5-0e4e7465bb99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eab6da6450c54419a334c451862c0b96'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:11.193025 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-da73d03d-c54e-4a8d-bcf5-0e4e7465bb99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eab6da6450c54419a334c451862c0b96'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:11.193802 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-da73d03d-c54e-4a8d-bcf5-0e4e7465bb99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eab6da6450c54419a334c451862c0b96'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:11.202066 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-da73d03d-c54e-4a8d-bcf5-0e4e7465bb99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eab6da6450c54419a334c451862c0b96'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:11.214238 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-da73d03d-c54e-4a8d-bcf5-0e4e7465bb99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eab6da6450c54419a334c451862c0b96'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:11.220240 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1033/2065] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:20:11 2022] DELETE /identity/v3/users/eab6da6450c54419a334c451862c0b96 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:20:11.425517 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d32e71-8726-44a5-908e-9d1b64611a0c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:11.427062 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-74d32e71-8726-44a5-908e-9d1b64611a0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:11.442753 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d32e71-8726-44a5-908e-9d1b64611a0c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:11.503099 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fda94b-c6cc-44fb-a910-5c19ff471a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:11.504202 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fda94b-c6cc-44fb-a910-5c19ff471a49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:11.504492 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fda94b-c6cc-44fb-a910-5c19ff471a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:11.504771 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fda94b-c6cc-44fb-a910-5c19ff471a49 admin admin] PATH_INFO: `/v3/users/f795bb8c30bc4a20a741ea914db1320a` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:11.506101 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fda94b-c6cc-44fb-a910-5c19ff471a49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f795bb8c30bc4a20a741ea914db1320a)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:11.508023 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fda94b-c6cc-44fb-a910-5c19ff471a49 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:11.515756 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:11.559810 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-50fda94b-c6cc-44fb-a910-5c19ff471a49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f795bb8c30bc4a20a741ea914db1320a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:11.560477 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-50fda94b-c6cc-44fb-a910-5c19ff471a49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f795bb8c30bc4a20a741ea914db1320a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:11.569426 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-50fda94b-c6cc-44fb-a910-5c19ff471a49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f795bb8c30bc4a20a741ea914db1320a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:11.584530 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-50fda94b-c6cc-44fb-a910-5c19ff471a49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f795bb8c30bc4a20a741ea914db1320a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:11.585539 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-50fda94b-c6cc-44fb-a910-5c19ff471a49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f795bb8c30bc4a20a741ea914db1320a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:11.600588 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1033/2066] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:20:11 2022] DELETE /identity/v3/users/f795bb8c30bc4a20a741ea914db1320a => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:20:11.677516 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da73d03d-c54e-4a8d-bcf5-0e4e7465bb99 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:11.679153 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-da73d03d-c54e-4a8d-bcf5-0e4e7465bb99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:11.701177 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da73d03d-c54e-4a8d-bcf5-0e4e7465bb99 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:11.762293 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:11.769663 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b31d97-59ce-43b6-9d5b-91f3556ae1f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:11.770934 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b31d97-59ce-43b6-9d5b-91f3556ae1f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:11.771296 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b31d97-59ce-43b6-9d5b-91f3556ae1f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:11.771580 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b31d97-59ce-43b6-9d5b-91f3556ae1f4 admin admin] PATH_INFO: `/v3/projects/b030c2ed96da4869875c35e43964dc10` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:11.774119 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b31d97-59ce-43b6-9d5b-91f3556ae1f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b030c2ed96da4869875c35e43964dc10)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:11.776763 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b31d97-59ce-43b6-9d5b-91f3556ae1f4 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:11.804790 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fda94b-c6cc-44fb-a910-5c19ff471a49 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:11.806991 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-50fda94b-c6cc-44fb-a910-5c19ff471a49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:11.833643 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-70b31d97-59ce-43b6-9d5b-91f3556ae1f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b030c2ed96da4869875c35e43964dc10', 'request_id': 'req-70b31d97-59ce-43b6-9d5b-91f3556ae1f4'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:11.834182 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fda94b-c6cc-44fb-a910-5c19ff471a49 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:11.840119 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-70b31d97-59ce-43b6-9d5b-91f3556ae1f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b030c2ed96da4869875c35e43964dc10', 'request_id': 'req-70b31d97-59ce-43b6-9d5b-91f3556ae1f4'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:11.847175 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-70b31d97-59ce-43b6-9d5b-91f3556ae1f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b030c2ed96da4869875c35e43964dc10', 'request_id': 'req-70b31d97-59ce-43b6-9d5b-91f3556ae1f4'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:11.861478 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-70b31d97-59ce-43b6-9d5b-91f3556ae1f4 admin admin] The token cache is being invalidate because project b030c2ed96da4869875c35e43964dc10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:20:11.861770 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-70b31d97-59ce-43b6-9d5b-91f3556ae1f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:11.862609 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-70b31d97-59ce-43b6-9d5b-91f3556ae1f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:11.864366 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1034/2067] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:20:11 2022] DELETE /identity/v3/projects/b030c2ed96da4869875c35e43964dc10 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:20:11.869626 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a492a40-23c6-477b-b94d-8d02bd2f3b68 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:11.869919 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a492a40-23c6-477b-b94d-8d02bd2f3b68 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:11.870204 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a492a40-23c6-477b-b94d-8d02bd2f3b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:11.889114 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-9a492a40-23c6-477b-b94d-8d02bd2f3b68 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:11.890350 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-9a492a40-23c6-477b-b94d-8d02bd2f3b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:11.898589 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c511b548-8eee-4290-b788-2b7ed4a51dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:11.899601 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c511b548-8eee-4290-b788-2b7ed4a51dea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:11.899877 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c511b548-8eee-4290-b788-2b7ed4a51dea admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:11.900187 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c511b548-8eee-4290-b788-2b7ed4a51dea admin admin] PATH_INFO: `/v3/projects/f571f9a1b0d04239be18c627f5a985e2` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:11.901416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c511b548-8eee-4290-b788-2b7ed4a51dea admin admin] RBAC: Authorizing `identity:delete_project(project_id=f571f9a1b0d04239be18c627f5a985e2)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:11.903317 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c511b548-8eee-4290-b788-2b7ed4a51dea admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:11.957697 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1035/2068] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:20:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:20:11.958354 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c511b548-8eee-4290-b788-2b7ed4a51dea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f571f9a1b0d04239be18c627f5a985e2', 'request_id': 'req-c511b548-8eee-4290-b788-2b7ed4a51dea'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:11.966552 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c71147d-aaf2-419e-b18c-0747ec070f87 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:11.966787 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c71147d-aaf2-419e-b18c-0747ec070f87 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:11.967124 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c71147d-aaf2-419e-b18c-0747ec070f87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:11.969517 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c511b548-8eee-4290-b788-2b7ed4a51dea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f571f9a1b0d04239be18c627f5a985e2', 'request_id': 'req-c511b548-8eee-4290-b788-2b7ed4a51dea'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:11.978212 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c511b548-8eee-4290-b788-2b7ed4a51dea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f571f9a1b0d04239be18c627f5a985e2', 'request_id': 'req-c511b548-8eee-4290-b788-2b7ed4a51dea'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:11.985706 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-9c71147d-aaf2-419e-b18c-0747ec070f87 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:11.986942 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-9c71147d-aaf2-419e-b18c-0747ec070f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:11.994706 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c511b548-8eee-4290-b788-2b7ed4a51dea admin admin] The token cache is being invalidate because project f571f9a1b0d04239be18c627f5a985e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:20:11.995394 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c511b548-8eee-4290-b788-2b7ed4a51dea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:11.996105 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c511b548-8eee-4290-b788-2b7ed4a51dea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:11.998470 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1034/2069] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:20:11 2022] DELETE /identity/v3/projects/f571f9a1b0d04239be18c627f5a985e2 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:20:12.002298 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c511b548-8eee-4290-b788-2b7ed4a51dea admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:12.003772 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c511b548-8eee-4290-b788-2b7ed4a51dea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:12.026081 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:12.050245 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1036/2070] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:20:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:20:12.054361 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c71147d-aaf2-419e-b18c-0747ec070f87 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:12.063763 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c71147d-aaf2-419e-b18c-0747ec070f87 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:12.068803 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b596d-8144-475f-afaa-834f83c9f4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:12.069321 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b596d-8144-475f-afaa-834f83c9f4ed admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:12.069608 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b596d-8144-475f-afaa-834f83c9f4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:12.069889 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b596d-8144-475f-afaa-834f83c9f4ed admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:12.070640 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8b596d-8144-475f-afaa-834f83c9f4ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:20:12.071015 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8b596d-8144-475f-afaa-834f83c9f4ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:12.072687 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8b596d-8144-475f-afaa-834f83c9f4ed admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:12.082105 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c511b548-8eee-4290-b788-2b7ed4a51dea admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:12.084256 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1037/2071] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:20:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:20:12.088000 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fc3d8a-5c2c-4e28-8bd1-9f8ad3753995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:12.089066 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b596d-8144-475f-afaa-834f83c9f4ed admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:12.089340 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc3d8a-5c2c-4e28-8bd1-9f8ad3753995 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:12.089692 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc3d8a-5c2c-4e28-8bd1-9f8ad3753995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:12.090051 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc3d8a-5c2c-4e28-8bd1-9f8ad3753995 admin admin] PATH_INFO: `/v3/users/f645f4a8ac2243da8bcc989cdc59c309` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:12.091350 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fc3d8a-5c2c-4e28-8bd1-9f8ad3753995 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f645f4a8ac2243da8bcc989cdc59c309)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:12.093495 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fc3d8a-5c2c-4e28-8bd1-9f8ad3753995 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:12.098862 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b596d-8144-475f-afaa-834f83c9f4ed admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:12.104047 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18685f6f-8d6a-49c8-9cba-f2a091f2159f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:12.104686 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18685f6f-8d6a-49c8-9cba-f2a091f2159f admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:12.104978 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18685f6f-8d6a-49c8-9cba-f2a091f2159f admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:12.105290 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18685f6f-8d6a-49c8-9cba-f2a091f2159f admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:12.105977 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18685f6f-8d6a-49c8-9cba-f2a091f2159f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:12.107867 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18685f6f-8d6a-49c8-9cba-f2a091f2159f admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:12.121131 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:12.142089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1038/2072] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:20:12 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:20:12.143108 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18685f6f-8d6a-49c8-9cba-f2a091f2159f admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:12.144802 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-49fc3d8a-5c2c-4e28-8bd1-9f8ad3753995 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f645f4a8ac2243da8bcc989cdc59c309'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:12.145601 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-49fc3d8a-5c2c-4e28-8bd1-9f8ad3753995 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f645f4a8ac2243da8bcc989cdc59c309'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:12.154315 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-49fc3d8a-5c2c-4e28-8bd1-9f8ad3753995 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f645f4a8ac2243da8bcc989cdc59c309'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:12.167466 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-49fc3d8a-5c2c-4e28-8bd1-9f8ad3753995 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f645f4a8ac2243da8bcc989cdc59c309'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:12.168152 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-49fc3d8a-5c2c-4e28-8bd1-9f8ad3753995 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f645f4a8ac2243da8bcc989cdc59c309'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:12.181409 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1035/2073] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:20:12 2022] DELETE /identity/v3/users/f645f4a8ac2243da8bcc989cdc59c309 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:20:12.212400 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18685f6f-8d6a-49c8-9cba-f2a091f2159f admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:12.277459 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2c9cac-3374-4703-89ac-7f902ca32bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:12.278136 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2c9cac-3374-4703-89ac-7f902ca32bf0 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:12.278417 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2c9cac-3374-4703-89ac-7f902ca32bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:12.278698 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2c9cac-3374-4703-89ac-7f902ca32bf0 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:12.279388 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2c9cac-3374-4703-89ac-7f902ca32bf0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:12.281377 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2c9cac-3374-4703-89ac-7f902ca32bf0 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:12.303194 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1039/2074] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:20:12 2022] POST /identity/v3/users => generated 288 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:20:12.307723 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fc3d8a-5c2c-4e28-8bd1-9f8ad3753995 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:12.309254 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-49fc3d8a-5c2c-4e28-8bd1-9f8ad3753995 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:12.322157 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fc3d8a-5c2c-4e28-8bd1-9f8ad3753995 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:12.326982 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b81ab9-161b-43f3-b554-9c2755e6cfd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:12.327735 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b81ab9-161b-43f3-b554-9c2755e6cfd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:12.328255 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b81ab9-161b-43f3-b554-9c2755e6cfd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:12.328601 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b81ab9-161b-43f3-b554-9c2755e6cfd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:12.329436 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b81ab9-161b-43f3-b554-9c2755e6cfd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:12.331212 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b81ab9-161b-43f3-b554-9c2755e6cfd9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:12.343086 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1036/2075] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:20:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:20:12.348494 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2c9cac-3374-4703-89ac-7f902ca32bf0 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:12.357586 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b81ab9-161b-43f3-b554-9c2755e6cfd9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:12.360090 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f8b81ab9-161b-43f3-b554-9c2755e6cfd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:12.363086 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2c9cac-3374-4703-89ac-7f902ca32bf0 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:12.369475 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc000e6d-05ff-4f74-95e2-cc0f17c82e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:12.370159 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc000e6d-05ff-4f74-95e2-cc0f17c82e04 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:12.370449 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc000e6d-05ff-4f74-95e2-cc0f17c82e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:12.370733 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc000e6d-05ff-4f74-95e2-cc0f17c82e04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:12.371468 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc000e6d-05ff-4f74-95e2-cc0f17c82e04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:12.373121 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc000e6d-05ff-4f74-95e2-cc0f17c82e04 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:12.377576 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b81ab9-161b-43f3-b554-9c2755e6cfd9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:12.386322 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1040/2076] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:20:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:20:12.389284 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe11189-12e7-4373-b662-3544e581ab19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:12.390321 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc000e6d-05ff-4f74-95e2-cc0f17c82e04 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:12.393466 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe11189-12e7-4373-b662-3544e581ab19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:12.395299 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe11189-12e7-4373-b662-3544e581ab19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:12.395677 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe11189-12e7-4373-b662-3544e581ab19 admin admin] PATH_INFO: `/v3/projects/6b42826f396a4391bef96910cfb8e1fb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:12.398470 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe11189-12e7-4373-b662-3544e581ab19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b42826f396a4391bef96910cfb8e1fb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:12.401384 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe11189-12e7-4373-b662-3544e581ab19 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:12.405212 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc000e6d-05ff-4f74-95e2-cc0f17c82e04 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:12.410623 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4bba0d-24f6-4a14-973e-06cdeae76ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:12.411649 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4bba0d-24f6-4a14-973e-06cdeae76ee0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:12.412038 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4bba0d-24f6-4a14-973e-06cdeae76ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:12.412424 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4bba0d-24f6-4a14-973e-06cdeae76ee0 admin admin] PATH_INFO: `/v3/projects/c10c780b03114d81b50917c620f44de8/users/c7a9c31229954717b2be14a299fb2612/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:12.428714 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4bba0d-24f6-4a14-973e-06cdeae76ee0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c10c780b03114d81b50917c620f44de8, user_id=c7a9c31229954717b2be14a299fb2612, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:12.432848 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4bba0d-24f6-4a14-973e-06cdeae76ee0 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:12.446293 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1041/2077] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:20:12 2022] PUT /identity/v3/projects/c10c780b03114d81b50917c620f44de8/users/c7a9c31229954717b2be14a299fb2612/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:20:12.454345 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-abe11189-12e7-4373-b662-3544e581ab19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b42826f396a4391bef96910cfb8e1fb', 'request_id': 'req-abe11189-12e7-4373-b662-3544e581ab19'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:12.456308 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcac4f5-53b2-4c43-9ec6-fb38a4baecee None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:12.457346 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcac4f5-53b2-4c43-9ec6-fb38a4baecee None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:12.457643 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcac4f5-53b2-4c43-9ec6-fb38a4baecee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:12.460030 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-abe11189-12e7-4373-b662-3544e581ab19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b42826f396a4391bef96910cfb8e1fb', 'request_id': 'req-abe11189-12e7-4373-b662-3544e581ab19'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:12.468344 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-abe11189-12e7-4373-b662-3544e581ab19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b42826f396a4391bef96910cfb8e1fb', 'request_id': 'req-abe11189-12e7-4373-b662-3544e581ab19'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:12.481712 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-abe11189-12e7-4373-b662-3544e581ab19 admin admin] The token cache is being invalidate because project 6b42826f396a4391bef96910cfb8e1fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:20:12.482005 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-abe11189-12e7-4373-b662-3544e581ab19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:12.482693 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-abe11189-12e7-4373-b662-3544e581ab19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:12.484281 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1037/2078] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:20:12 2022] DELETE /identity/v3/projects/6b42826f396a4391bef96910cfb8e1fb => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:20:12.486789 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-7dcac4f5-53b2-4c43-9ec6-fb38a4baecee None None] MFA Rules not processed for user `c7a9c31229954717b2be14a299fb2612`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:12.488318 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7dcac4f5-53b2-4c43-9ec6-fb38a4baecee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:12.490701 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b5331-0383-4a49-a4ee-292be27e835b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:12.490994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b5331-0383-4a49-a4ee-292be27e835b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:12.491270 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b5331-0383-4a49-a4ee-292be27e835b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:12.499037 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:12.511248 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-071b5331-0383-4a49-a4ee-292be27e835b None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:12.513620 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-071b5331-0383-4a49-a4ee-292be27e835b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:12.533608 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:12.560161 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1042/2079] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:20:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:20:12.564686 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcac4f5-53b2-4c43-9ec6-fb38a4baecee None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:12.566063 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7dcac4f5-53b2-4c43-9ec6-fb38a4baecee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:12.581069 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1038/2080] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:20:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:20:12.591099 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf890d-bc9f-4829-974e-db1ab77971e0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:12.591388 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf890d-bc9f-4829-974e-db1ab77971e0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:12.591667 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf890d-bc9f-4829-974e-db1ab77971e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:12.612893 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a8cf890d-bc9f-4829-974e-db1ab77971e0 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:12.614523 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a8cf890d-bc9f-4829-974e-db1ab77971e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:12.623398 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1039/2081] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:20:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:20:12.627331 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cf890d-bc9f-4829-974e-db1ab77971e0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:12.637079 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cf890d-bc9f-4829-974e-db1ab77971e0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:12.641858 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687ebead-71b8-4d9a-857b-bf633e444882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:12.641991 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcac4f5-53b2-4c43-9ec6-fb38a4baecee None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:12.642508 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ebead-71b8-4d9a-857b-bf633e444882 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:12.642793 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ebead-71b8-4d9a-857b-bf633e444882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:12.643073 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ebead-71b8-4d9a-857b-bf633e444882 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:12.643798 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687ebead-71b8-4d9a-857b-bf633e444882 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:20:12.644108 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687ebead-71b8-4d9a-857b-bf633e444882 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:12.645732 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687ebead-71b8-4d9a-857b-bf633e444882 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:12.646631 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b2c38d-c75f-47cc-9a98-d60e9646cb78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:12.647287 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2c38d-c75f-47cc-9a98-d60e9646cb78 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:12.647576 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2c38d-c75f-47cc-9a98-d60e9646cb78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:12.647856 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2c38d-c75f-47cc-9a98-d60e9646cb78 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:12.648552 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b2c38d-c75f-47cc-9a98-d60e9646cb78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:12.650611 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b2c38d-c75f-47cc-9a98-d60e9646cb78 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:12.660943 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1040/2082] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:20:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:20:12.661953 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687ebead-71b8-4d9a-857b-bf633e444882 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:12.671886 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687ebead-71b8-4d9a-857b-bf633e444882 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:12.676536 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1a6b72-9b91-4d2b-b416-71c57063ba7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:12.680569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1043/2083] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:20:12 2022] POST /identity/v3/projects => generated 377 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:20:12.680781 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a6b72-9b91-4d2b-b416-71c57063ba7b admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:12.680781 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a6b72-9b91-4d2b-b416-71c57063ba7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:12.680781 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a6b72-9b91-4d2b-b416-71c57063ba7b admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:12.680781 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1a6b72-9b91-4d2b-b416-71c57063ba7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:12.681095 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1a6b72-9b91-4d2b-b416-71c57063ba7b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:12.682163 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b2c38d-c75f-47cc-9a98-d60e9646cb78 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:12.708283 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1041/2084] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:20:12 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:20:12.713023 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1a6b72-9b91-4d2b-b416-71c57063ba7b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:12.747014 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b2c38d-c75f-47cc-9a98-d60e9646cb78 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:12.777196 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1a6b72-9b91-4d2b-b416-71c57063ba7b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:12.782613 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe7bdbb-e3a4-4657-b623-27e49135ccc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:12.783291 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe7bdbb-e3a4-4657-b623-27e49135ccc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:12.783579 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe7bdbb-e3a4-4657-b623-27e49135ccc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:12.783864 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe7bdbb-e3a4-4657-b623-27e49135ccc4 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:12.784561 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe7bdbb-e3a4-4657-b623-27e49135ccc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:12.786512 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe7bdbb-e3a4-4657-b623-27e49135ccc4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:12.810934 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1042/2085] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:20:12 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:20:12.811974 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe7bdbb-e3a4-4657-b623-27e49135ccc4 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:12.817096 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46166450-5548-49f8-8c3d-ebf3c77700b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:12.817777 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46166450-5548-49f8-8c3d-ebf3c77700b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:12.818060 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46166450-5548-49f8-8c3d-ebf3c77700b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:12.818340 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46166450-5548-49f8-8c3d-ebf3c77700b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:12.819034 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46166450-5548-49f8-8c3d-ebf3c77700b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:12.821690 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46166450-5548-49f8-8c3d-ebf3c77700b7 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:12.822545 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe7bdbb-e3a4-4657-b623-27e49135ccc4 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:12.827651 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44d0f61-1cd7-4d88-8ee8-13643f00b63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:12.828748 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d0f61-1cd7-4d88-8ee8-13643f00b63c admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:12.829290 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d0f61-1cd7-4d88-8ee8-13643f00b63c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:12.829797 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d0f61-1cd7-4d88-8ee8-13643f00b63c admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:12.831184 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44d0f61-1cd7-4d88-8ee8-13643f00b63c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:12.834365 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44d0f61-1cd7-4d88-8ee8-13643f00b63c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:12.849095 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1044/2086] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:20:12 2022] POST /identity/v3/users => generated 289 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:20:12.851140 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1043/2087] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:20:12 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:20:12.852622 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46166450-5548-49f8-8c3d-ebf3c77700b7 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:12.854110 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44d0f61-1cd7-4d88-8ee8-13643f00b63c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:12.860387 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:12.866654 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46166450-5548-49f8-8c3d-ebf3c77700b7 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:12.867362 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44d0f61-1cd7-4d88-8ee8-13643f00b63c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:12.872283 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7cd89d-45ce-44cf-834d-be63297c0527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:12.872951 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7cd89d-45ce-44cf-834d-be63297c0527 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:12.873274 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7cd89d-45ce-44cf-834d-be63297c0527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:12.873555 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7cd89d-45ce-44cf-834d-be63297c0527 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:12.873905 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6ea8e8-5b81-4133-9d21-6ed810641141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:12.874285 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7cd89d-45ce-44cf-834d-be63297c0527 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:12.874630 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ea8e8-5b81-4133-9d21-6ed810641141 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:12.874917 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ea8e8-5b81-4133-9d21-6ed810641141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:12.875197 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ea8e8-5b81-4133-9d21-6ed810641141 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:12.875878 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7cd89d-45ce-44cf-834d-be63297c0527 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:12.875991 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6ea8e8-5b81-4133-9d21-6ed810641141 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:12.877609 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6ea8e8-5b81-4133-9d21-6ed810641141 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:12.887373 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1044/2088] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:20:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:20:12.888675 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1045/2089] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:20:12 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:20:12.892595 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6ea8e8-5b81-4133-9d21-6ed810641141 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:12.892723 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7cd89d-45ce-44cf-834d-be63297c0527 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:12.902889 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7cd89d-45ce-44cf-834d-be63297c0527 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:12.903458 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6ea8e8-5b81-4133-9d21-6ed810641141 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:12.907655 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ace96b-fa56-440b-a9d2-7f0090a02072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:12.908062 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac863b3-e424-420a-80ee-13a208bca78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:12.908527 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ace96b-fa56-440b-a9d2-7f0090a02072 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:12.908701 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac863b3-e424-420a-80ee-13a208bca78c admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:12.908813 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ace96b-fa56-440b-a9d2-7f0090a02072 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:12.908987 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac863b3-e424-420a-80ee-13a208bca78c admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:12.909111 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ace96b-fa56-440b-a9d2-7f0090a02072 admin admin] PATH_INFO: `/v3/projects/6d3fa60f618049c58178c8f0d39d6fc1/users/6158c8a893534b259a2031496dd70ad7/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:12.909298 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac863b3-e424-420a-80ee-13a208bca78c admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:12.910014 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac863b3-e424-420a-80ee-13a208bca78c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:12.911606 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac863b3-e424-420a-80ee-13a208bca78c admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:12.922903 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1046/2090] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:20:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:20:12.924510 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ace96b-fa56-440b-a9d2-7f0090a02072 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d3fa60f618049c58178c8f0d39d6fc1, user_id=6158c8a893534b259a2031496dd70ad7, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:12.927180 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac863b3-e424-420a-80ee-13a208bca78c admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:12.927394 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ace96b-fa56-440b-a9d2-7f0090a02072 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:12.937471 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac863b3-e424-420a-80ee-13a208bca78c admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:12.941312 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1045/2091] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:20:12 2022] PUT /identity/v3/projects/6d3fa60f618049c58178c8f0d39d6fc1/users/6158c8a893534b259a2031496dd70ad7/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:20:12.952235 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7474a-446f-4e85-b454-021124a8a882 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:12.952533 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7474a-446f-4e85-b454-021124a8a882 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:12.952819 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7474a-446f-4e85-b454-021124a8a882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:12.965631 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:12.981555 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d0b7474a-446f-4e85-b454-021124a8a882 None None] MFA Rules not processed for user `6158c8a893534b259a2031496dd70ad7`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:12.982791 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d0b7474a-446f-4e85-b454-021124a8a882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:13.001658 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c71e9a-b737-4dbf-995e-a378d1252624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:13.002476 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c71e9a-b737-4dbf-995e-a378d1252624 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:13.003049 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c71e9a-b737-4dbf-995e-a378d1252624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:13.003345 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c71e9a-b737-4dbf-995e-a378d1252624 admin admin] PATH_INFO: `/v3/projects/05da1a497c484ef68715eb405e664ee3/users/4b1e677f4a93483d93ae754ecf8cfb16/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:13.029315 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c71e9a-b737-4dbf-995e-a378d1252624 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05da1a497c484ef68715eb405e664ee3, user_id=4b1e677f4a93483d93ae754ecf8cfb16, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:13.031709 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c71e9a-b737-4dbf-995e-a378d1252624 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:13.047981 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1047/2092] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:20:12 2022] PUT /identity/v3/projects/05da1a497c484ef68715eb405e664ee3/users/4b1e677f4a93483d93ae754ecf8cfb16/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:20:13.059098 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f91433-7767-4f8e-99f0-003ca7158546 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:13.059629 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f91433-7767-4f8e-99f0-003ca7158546 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:13.059760 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1046/2093] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:20:12 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:20:13.059858 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f91433-7767-4f8e-99f0-003ca7158546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:13.071795 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b7474a-446f-4e85-b454-021124a8a882 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:13.073332 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d0b7474a-446f-4e85-b454-021124a8a882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:13.103008 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-14f91433-7767-4f8e-99f0-003ca7158546 None None] MFA Rules not processed for user `4b1e677f4a93483d93ae754ecf8cfb16`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:13.104693 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-14f91433-7767-4f8e-99f0-003ca7158546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:13.132344 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b7474a-446f-4e85-b454-021124a8a882 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:13.173085 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1048/2094] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:20:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:20:13.177515 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f91433-7767-4f8e-99f0-003ca7158546 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:13.192846 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4033f906-579b-42e3-908f-0e1a7ea8a552 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:13.193737 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4033f906-579b-42e3-908f-0e1a7ea8a552 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:13.194258 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4033f906-579b-42e3-908f-0e1a7ea8a552 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:13.194537 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4033f906-579b-42e3-908f-0e1a7ea8a552 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:13.243246 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:13.250003 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f91433-7767-4f8e-99f0-003ca7158546 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:13.255747 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e50780-573c-4593-a2a4-d452351c824d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:13.256427 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e50780-573c-4593-a2a4-d452351c824d admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:13.256735 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e50780-573c-4593-a2a4-d452351c824d admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:13.257011 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e50780-573c-4593-a2a4-d452351c824d admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:13.257769 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e50780-573c-4593-a2a4-d452351c824d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:13.259763 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e50780-573c-4593-a2a4-d452351c824d admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:13.266225 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4033f906-579b-42e3-908f-0e1a7ea8a552 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:13.268048 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4033f906-579b-42e3-908f-0e1a7ea8a552 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:13.277931 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1047/2095] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:20:13 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:20:13.289497 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1049/2096] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:20:13 2022] POST /identity/v3/projects => generated 377 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:20:13.294287 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4033f906-579b-42e3-908f-0e1a7ea8a552 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:13.303757 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e50780-573c-4593-a2a4-d452351c824d admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:13.305848 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4033f906-579b-42e3-908f-0e1a7ea8a552 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:13.309547 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:13.362931 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5c4343-2a55-4df8-8b90-aa715fe9adfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:13.363651 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5c4343-2a55-4df8-8b90-aa715fe9adfe admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:13.363937 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5c4343-2a55-4df8-8b90-aa715fe9adfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:13.364213 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5c4343-2a55-4df8-8b90-aa715fe9adfe admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:13.365163 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5c4343-2a55-4df8-8b90-aa715fe9adfe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:13.367203 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5c4343-2a55-4df8-8b90-aa715fe9adfe admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:13.368976 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e50780-573c-4593-a2a4-d452351c824d admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:13.373628 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf26d9e-8a96-43c0-b47d-0a8cd88fa2e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:13.374494 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf26d9e-8a96-43c0-b47d-0a8cd88fa2e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:13.374777 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf26d9e-8a96-43c0-b47d-0a8cd88fa2e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:13.375080 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf26d9e-8a96-43c0-b47d-0a8cd88fa2e8 admin admin] PATH_INFO: `/v3/users/6158c8a893534b259a2031496dd70ad7` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:13.376301 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf26d9e-8a96-43c0-b47d-0a8cd88fa2e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6158c8a893534b259a2031496dd70ad7)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:13.378259 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf26d9e-8a96-43c0-b47d-0a8cd88fa2e8 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:13.392266 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1048/2097] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:20:13 2022] POST /identity/v3/users => generated 295 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:20:13.394431 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5c4343-2a55-4df8-8b90-aa715fe9adfe admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:13.405073 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5c4343-2a55-4df8-8b90-aa715fe9adfe admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:13.409715 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d9c28-2dbf-42ee-bcbb-b0a31938eb99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:13.410372 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d9c28-2dbf-42ee-bcbb-b0a31938eb99 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:13.410651 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d9c28-2dbf-42ee-bcbb-b0a31938eb99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:13.410928 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d9c28-2dbf-42ee-bcbb-b0a31938eb99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:13.411647 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961d9c28-2dbf-42ee-bcbb-b0a31938eb99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:13.413579 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961d9c28-2dbf-42ee-bcbb-b0a31938eb99 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:13.426023 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1049/2098] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:20:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:20:13.429906 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d9c28-2dbf-42ee-bcbb-b0a31938eb99 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:13.437941 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-8cf26d9e-8a96-43c0-b47d-0a8cd88fa2e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6158c8a893534b259a2031496dd70ad7'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:13.441820 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d9c28-2dbf-42ee-bcbb-b0a31938eb99 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:13.445657 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-8cf26d9e-8a96-43c0-b47d-0a8cd88fa2e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6158c8a893534b259a2031496dd70ad7'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:13.447421 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dabbdaa-7da7-4b49-8b25-f9b16d3300fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:13.447593 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-8cf26d9e-8a96-43c0-b47d-0a8cd88fa2e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6158c8a893534b259a2031496dd70ad7'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:13.448442 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dabbdaa-7da7-4b49-8b25-f9b16d3300fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:13.448716 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dabbdaa-7da7-4b49-8b25-f9b16d3300fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:13.448993 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dabbdaa-7da7-4b49-8b25-f9b16d3300fc admin admin] PATH_INFO: `/v3/projects/8359e756bdc94df7a7afc572bddde935/users/128161c188d74caa98caa4822e9d2661/roles/f36e053b442540f08041ac7878760f65` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:13.455469 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-8cf26d9e-8a96-43c0-b47d-0a8cd88fa2e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6158c8a893534b259a2031496dd70ad7'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:13.466029 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dabbdaa-7da7-4b49-8b25-f9b16d3300fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=8359e756bdc94df7a7afc572bddde935, user_id=128161c188d74caa98caa4822e9d2661, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:13.468681 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dabbdaa-7da7-4b49-8b25-f9b16d3300fc admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:13.470337 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-8cf26d9e-8a96-43c0-b47d-0a8cd88fa2e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6158c8a893534b259a2031496dd70ad7'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:13.476779 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1050/2099] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:20:13 2022] DELETE /identity/v3/users/6158c8a893534b259a2031496dd70ad7 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:20:13.481886 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea919dec-4e1f-423c-b657-cbbd19b0b0c9 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:13.482202 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea919dec-4e1f-423c-b657-cbbd19b0b0c9 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:13.482403 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1050/2100] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:20:13 2022] PUT /identity/v3/projects/8359e756bdc94df7a7afc572bddde935/users/128161c188d74caa98caa4822e9d2661/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:20:13.482512 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea919dec-4e1f-423c-b657-cbbd19b0b0c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:13.485913 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dabbdaa-7da7-4b49-8b25-f9b16d3300fc admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:13.487534 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0dabbdaa-7da7-4b49-8b25-f9b16d3300fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:13.500117 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-ea919dec-4e1f-423c-b657-cbbd19b0b0c9 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:13.501390 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ea919dec-4e1f-423c-b657-cbbd19b0b0c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:13.553064 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:13.558114 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dabbdaa-7da7-4b49-8b25-f9b16d3300fc admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:13.568843 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1051/2101] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:20:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:20:13.579698 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea919dec-4e1f-423c-b657-cbbd19b0b0c9 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:13.581532 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ea919dec-4e1f-423c-b657-cbbd19b0b0c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:13.625624 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb61dd-54fc-4471-ba4a-9a2aec58cec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:13.626417 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb61dd-54fc-4471-ba4a-9a2aec58cec6 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:13.626699 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb61dd-54fc-4471-ba4a-9a2aec58cec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:13.626978 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb61dd-54fc-4471-ba4a-9a2aec58cec6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:13.627702 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bb61dd-54fc-4471-ba4a-9a2aec58cec6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:13.629356 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bb61dd-54fc-4471-ba4a-9a2aec58cec6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:13.643758 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1051/2102] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:20:13 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:20:13.644197 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea919dec-4e1f-423c-b657-cbbd19b0b0c9 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:13.647314 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb61dd-54fc-4471-ba4a-9a2aec58cec6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:13.648208 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd50b61-1dcb-49f2-87b9-28dd268a096e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:13.649533 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd50b61-1dcb-49f2-87b9-28dd268a096e service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:13.649796 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd50b61-1dcb-49f2-87b9-28dd268a096e service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:13.650150 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd50b61-1dcb-49f2-87b9-28dd268a096e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:13.658309 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:13.662648 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb61dd-54fc-4471-ba4a-9a2aec58cec6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:13.670107 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd50b61-1dcb-49f2-87b9-28dd268a096e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:13.670925 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5915e3ce-cb84-4993-a1f0-492a3607b2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:13.671574 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd50b61-1dcb-49f2-87b9-28dd268a096e service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:13.671948 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915e3ce-cb84-4993-a1f0-492a3607b2f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:13.672306 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915e3ce-cb84-4993-a1f0-492a3607b2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:13.672657 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915e3ce-cb84-4993-a1f0-492a3607b2f1 admin admin] PATH_INFO: `/v3/projects/8359e756bdc94df7a7afc572bddde935/users/128161c188d74caa98caa4822e9d2661/roles/f36e053b442540f08041ac7878760f65` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:13.674869 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5915e3ce-cb84-4993-a1f0-492a3607b2f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8359e756bdc94df7a7afc572bddde935, user_id=128161c188d74caa98caa4822e9d2661, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:13.678522 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5915e3ce-cb84-4993-a1f0-492a3607b2f1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:13.683413 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1052/2103] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:20:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:20:13.690941 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1052/2104] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:20:13 2022] PUT /identity/v3/projects/8359e756bdc94df7a7afc572bddde935/users/128161c188d74caa98caa4822e9d2661/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:20:13.703288 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77edcc3-9562-42c0-ad52-344baa0216ed None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:13.703862 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77edcc3-9562-42c0-ad52-344baa0216ed None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:13.704311 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77edcc3-9562-42c0-ad52-344baa0216ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:13.738812 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-c77edcc3-9562-42c0-ad52-344baa0216ed None None] MFA Rules not processed for user `128161c188d74caa98caa4822e9d2661`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:13.740254 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c77edcc3-9562-42c0-ad52-344baa0216ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:13.845922 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1053/2105] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:20:13 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:20:13.859756 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5915e3ce-cb84-4993-a1f0-492a3607b2f1 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:13.861522 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5915e3ce-cb84-4993-a1f0-492a3607b2f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:13.925421 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5915e3ce-cb84-4993-a1f0-492a3607b2f1 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:13.930178 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb6d2c4-071c-46dd-b1b6-05c27fbd658f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:13.930809 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6d2c4-071c-46dd-b1b6-05c27fbd658f service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:13.931097 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6d2c4-071c-46dd-b1b6-05c27fbd658f service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:13.931532 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6d2c4-071c-46dd-b1b6-05c27fbd658f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:13.933581 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bdb6d2c4-071c-46dd-b1b6-05c27fbd658f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:13.995121 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77edcc3-9562-42c0-ad52-344baa0216ed None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:13.996576 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c77edcc3-9562-42c0-ad52-344baa0216ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:14.021233 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb6d2c4-071c-46dd-b1b6-05c27fbd658f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:14.023102 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb6d2c4-071c-46dd-b1b6-05c27fbd658f service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:14.035978 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1053/2106] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:20:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:20:14.107758 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77edcc3-9562-42c0-ad52-344baa0216ed None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:14.112530 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d380e5-4035-4c80-8c20-2bd78b26361f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:14.113443 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d380e5-4035-4c80-8c20-2bd78b26361f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:14.113720 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d380e5-4035-4c80-8c20-2bd78b26361f admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:14.113996 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d380e5-4035-4c80-8c20-2bd78b26361f admin admin] PATH_INFO: `/v3/projects/6d3fa60f618049c58178c8f0d39d6fc1` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:14.115340 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d380e5-4035-4c80-8c20-2bd78b26361f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d3fa60f618049c58178c8f0d39d6fc1)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:14.117294 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d380e5-4035-4c80-8c20-2bd78b26361f admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:14.166360 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:14.197498 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-54d380e5-4035-4c80-8c20-2bd78b26361f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d3fa60f618049c58178c8f0d39d6fc1', 'request_id': 'req-54d380e5-4035-4c80-8c20-2bd78b26361f'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:14.204570 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-54d380e5-4035-4c80-8c20-2bd78b26361f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d3fa60f618049c58178c8f0d39d6fc1', 'request_id': 'req-54d380e5-4035-4c80-8c20-2bd78b26361f'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:14.212221 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-54d380e5-4035-4c80-8c20-2bd78b26361f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d3fa60f618049c58178c8f0d39d6fc1', 'request_id': 'req-54d380e5-4035-4c80-8c20-2bd78b26361f'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:14.227563 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-54d380e5-4035-4c80-8c20-2bd78b26361f admin admin] The token cache is being invalidate because project 6d3fa60f618049c58178c8f0d39d6fc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:20:14.228482 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-54d380e5-4035-4c80-8c20-2bd78b26361f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:14.229297 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-54d380e5-4035-4c80-8c20-2bd78b26361f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:14.236548 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1054/2107] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:20:13 2022] DELETE /identity/v3/projects/6d3fa60f618049c58178c8f0d39d6fc1 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:20:15.641195 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb6d2c4-071c-46dd-b1b6-05c27fbd658f service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:15.642774 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bdb6d2c4-071c-46dd-b1b6-05c27fbd658f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:15.654667 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb6d2c4-071c-46dd-b1b6-05c27fbd658f service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:15.706865 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e139579d-6798-48e7-8ae0-21c2c3a22199 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:15.707532 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e139579d-6798-48e7-8ae0-21c2c3a22199 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:15.707820 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e139579d-6798-48e7-8ae0-21c2c3a22199 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:15.708099 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e139579d-6798-48e7-8ae0-21c2c3a22199 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:15.710021 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e139579d-6798-48e7-8ae0-21c2c3a22199 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:15.770723 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e139579d-6798-48e7-8ae0-21c2c3a22199 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:15.772356 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e139579d-6798-48e7-8ae0-21c2c3a22199 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:15.777884 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:15.783878 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1054/2108] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:20:15 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:20:16.314459 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d380e5-4035-4c80-8c20-2bd78b26361f admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:16.315849 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-54d380e5-4035-4c80-8c20-2bd78b26361f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:16.328306 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d380e5-4035-4c80-8c20-2bd78b26361f admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:16.332745 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fe104f-f554-422b-9f20-dac93388b431 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:16.333532 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe104f-f554-422b-9f20-dac93388b431 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:16.333764 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe104f-f554-422b-9f20-dac93388b431 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:16.334039 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe104f-f554-422b-9f20-dac93388b431 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:16.336150 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a9fe104f-f554-422b-9f20-dac93388b431 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:16.423389 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fe104f-f554-422b-9f20-dac93388b431 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:16.425166 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fe104f-f554-422b-9f20-dac93388b431 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:16.437836 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1055/2109] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:20:16 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:20:21.053891 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e139579d-6798-48e7-8ae0-21c2c3a22199 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:21.057186 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e139579d-6798-48e7-8ae0-21c2c3a22199 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:21.134377 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e139579d-6798-48e7-8ae0-21c2c3a22199 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:21.138880 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dae72bd-b9da-42ce-9250-2257ffe7ec2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:21.139758 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae72bd-b9da-42ce-9250-2257ffe7ec2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:21.140013 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae72bd-b9da-42ce-9250-2257ffe7ec2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:21.140284 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae72bd-b9da-42ce-9250-2257ffe7ec2b admin admin] PATH_INFO: `/v3/users/290f50ecfbff4c98b0c0ac8661fdbadd` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:21.141607 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dae72bd-b9da-42ce-9250-2257ffe7ec2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=290f50ecfbff4c98b0c0ac8661fdbadd)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:21.143780 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dae72bd-b9da-42ce-9250-2257ffe7ec2b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:21.196684 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1dae72bd-b9da-42ce-9250-2257ffe7ec2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '290f50ecfbff4c98b0c0ac8661fdbadd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:21.197370 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1dae72bd-b9da-42ce-9250-2257ffe7ec2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '290f50ecfbff4c98b0c0ac8661fdbadd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:21.205008 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1dae72bd-b9da-42ce-9250-2257ffe7ec2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '290f50ecfbff4c98b0c0ac8661fdbadd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:21.217264 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1dae72bd-b9da-42ce-9250-2257ffe7ec2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '290f50ecfbff4c98b0c0ac8661fdbadd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:21.217930 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1dae72bd-b9da-42ce-9250-2257ffe7ec2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '290f50ecfbff4c98b0c0ac8661fdbadd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:21.235599 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1055/2110] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:20:21 2022] DELETE /identity/v3/users/290f50ecfbff4c98b0c0ac8661fdbadd => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:20:21.239740 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f3b44-7427-4716-8560-0225bad8e97e None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:21.240187 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f3b44-7427-4716-8560-0225bad8e97e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:21.240389 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f3b44-7427-4716-8560-0225bad8e97e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:21.258395 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-ba6f3b44-7427-4716-8560-0225bad8e97e None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:21.259466 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ba6f3b44-7427-4716-8560-0225bad8e97e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:21.339596 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1056/2111] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:20:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:20:21.349284 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dae72bd-b9da-42ce-9250-2257ffe7ec2b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:21.350797 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1dae72bd-b9da-42ce-9250-2257ffe7ec2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:21.422051 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dae72bd-b9da-42ce-9250-2257ffe7ec2b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:21.426080 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f000f6-3146-4f04-8df8-ac8fd3d20de8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:21.426733 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f000f6-3146-4f04-8df8-ac8fd3d20de8 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:21.427022 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f000f6-3146-4f04-8df8-ac8fd3d20de8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:21.427305 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f000f6-3146-4f04-8df8-ac8fd3d20de8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:21.438086 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f000f6-3146-4f04-8df8-ac8fd3d20de8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:21.439579 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f000f6-3146-4f04-8df8-ac8fd3d20de8 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:21.451486 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1056/2112] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:20:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:20:21.748317 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f3b44-7427-4716-8560-0225bad8e97e None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:21.749827 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ba6f3b44-7427-4716-8560-0225bad8e97e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:21.765854 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f3b44-7427-4716-8560-0225bad8e97e None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:21.771553 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac4b901-d2a2-4c6f-a796-9bb5cb7c41ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:21.772439 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4b901-d2a2-4c6f-a796-9bb5cb7c41ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:21.772981 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4b901-d2a2-4c6f-a796-9bb5cb7c41ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:21.773240 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4b901-d2a2-4c6f-a796-9bb5cb7c41ad admin admin] PATH_INFO: `/v3/projects/a58aafa018954fd4937d70682d6dc780` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:21.775207 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac4b901-d2a2-4c6f-a796-9bb5cb7c41ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=a58aafa018954fd4937d70682d6dc780)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:21.777577 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac4b901-d2a2-4c6f-a796-9bb5cb7c41ad admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:21.837617 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:21.837617 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-9ac4b901-d2a2-4c6f-a796-9bb5cb7c41ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a58aafa018954fd4937d70682d6dc780', 'request_id': 'req-9ac4b901-d2a2-4c6f-a796-9bb5cb7c41ad'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:21.840243 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-9ac4b901-d2a2-4c6f-a796-9bb5cb7c41ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a58aafa018954fd4937d70682d6dc780', 'request_id': 'req-9ac4b901-d2a2-4c6f-a796-9bb5cb7c41ad'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:21.847215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-9ac4b901-d2a2-4c6f-a796-9bb5cb7c41ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a58aafa018954fd4937d70682d6dc780', 'request_id': 'req-9ac4b901-d2a2-4c6f-a796-9bb5cb7c41ad'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:21.862177 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-9ac4b901-d2a2-4c6f-a796-9bb5cb7c41ad admin admin] The token cache is being invalidate because project a58aafa018954fd4937d70682d6dc780 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:20:21.862468 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-9ac4b901-d2a2-4c6f-a796-9bb5cb7c41ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:21.863130 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-9ac4b901-d2a2-4c6f-a796-9bb5cb7c41ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:21.864882 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1057/2113] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:20:21 2022] DELETE /identity/v3/projects/a58aafa018954fd4937d70682d6dc780 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:20:21.869479 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f000f6-3146-4f04-8df8-ac8fd3d20de8 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:21.880137 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f000f6-3146-4f04-8df8-ac8fd3d20de8 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:21.901405 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:21.942370 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533f4d4d-1605-4e7b-bbc1-da2feb4c93c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:21.943251 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f4d4d-1605-4e7b-bbc1-da2feb4c93c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:21.943538 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f4d4d-1605-4e7b-bbc1-da2feb4c93c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:21.943818 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f4d4d-1605-4e7b-bbc1-da2feb4c93c3 admin admin] PATH_INFO: `/v3/users/35f3dd6ab14c42f6928f09fa24c1ebe3` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:21.945082 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533f4d4d-1605-4e7b-bbc1-da2feb4c93c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35f3dd6ab14c42f6928f09fa24c1ebe3)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:21.947534 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533f4d4d-1605-4e7b-bbc1-da2feb4c93c3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:21.998343 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-533f4d4d-1605-4e7b-bbc1-da2feb4c93c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35f3dd6ab14c42f6928f09fa24c1ebe3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:21.999151 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-533f4d4d-1605-4e7b-bbc1-da2feb4c93c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35f3dd6ab14c42f6928f09fa24c1ebe3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:22.006333 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-533f4d4d-1605-4e7b-bbc1-da2feb4c93c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35f3dd6ab14c42f6928f09fa24c1ebe3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:22.018906 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-533f4d4d-1605-4e7b-bbc1-da2feb4c93c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35f3dd6ab14c42f6928f09fa24c1ebe3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:22.019642 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-533f4d4d-1605-4e7b-bbc1-da2feb4c93c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35f3dd6ab14c42f6928f09fa24c1ebe3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:22.038567 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1057/2114] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:20:21 2022] DELETE /identity/v3/users/35f3dd6ab14c42f6928f09fa24c1ebe3 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:20:22.328743 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac4b901-d2a2-4c6f-a796-9bb5cb7c41ad admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:22.330235 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-9ac4b901-d2a2-4c6f-a796-9bb5cb7c41ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:22.343584 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac4b901-d2a2-4c6f-a796-9bb5cb7c41ad admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:22.411025 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13af065-c9ab-4914-b503-5eab9a735acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:22.411992 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13af065-c9ab-4914-b503-5eab9a735acb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:22.412276 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13af065-c9ab-4914-b503-5eab9a735acb admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:22.412554 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13af065-c9ab-4914-b503-5eab9a735acb admin admin] PATH_INFO: `/v3/projects/d2b544835c5945dda00145d930dcedd3` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:22.413794 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13af065-c9ab-4914-b503-5eab9a735acb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2b544835c5945dda00145d930dcedd3)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:22.415750 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13af065-c9ab-4914-b503-5eab9a735acb admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:22.450458 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533f4d4d-1605-4e7b-bbc1-da2feb4c93c3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:22.451924 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-533f4d4d-1605-4e7b-bbc1-da2feb4c93c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:22.465015 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533f4d4d-1605-4e7b-bbc1-da2feb4c93c3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:22.469576 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f13af065-c9ab-4914-b503-5eab9a735acb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2b544835c5945dda00145d930dcedd3', 'request_id': 'req-f13af065-c9ab-4914-b503-5eab9a735acb'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:22.475516 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f13af065-c9ab-4914-b503-5eab9a735acb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2b544835c5945dda00145d930dcedd3', 'request_id': 'req-f13af065-c9ab-4914-b503-5eab9a735acb'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:22.482874 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f13af065-c9ab-4914-b503-5eab9a735acb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2b544835c5945dda00145d930dcedd3', 'request_id': 'req-f13af065-c9ab-4914-b503-5eab9a735acb'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:22.496267 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f13af065-c9ab-4914-b503-5eab9a735acb admin admin] The token cache is being invalidate because project d2b544835c5945dda00145d930dcedd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:20:22.496473 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f13af065-c9ab-4914-b503-5eab9a735acb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:22.497196 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f13af065-c9ab-4914-b503-5eab9a735acb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:22.499017 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1058/2115] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:20:22 2022] DELETE /identity/v3/projects/d2b544835c5945dda00145d930dcedd3 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:20:22.504594 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f4f75f-0380-44e8-8395-5fb47aebf8c6 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:22.504946 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f4f75f-0380-44e8-8395-5fb47aebf8c6 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:22.505325 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f4f75f-0380-44e8-8395-5fb47aebf8c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:22.523611 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-35f4f75f-0380-44e8-8395-5fb47aebf8c6 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:22.524825 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-35f4f75f-0380-44e8-8395-5fb47aebf8c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:22.527902 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b64573-394d-49db-a562-f92faff082cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:22.528773 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b64573-394d-49db-a562-f92faff082cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:22.529072 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b64573-394d-49db-a562-f92faff082cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:22.529352 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b64573-394d-49db-a562-f92faff082cd admin admin] PATH_INFO: `/v3/users/c7a9c31229954717b2be14a299fb2612` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:22.530551 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b64573-394d-49db-a562-f92faff082cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7a9c31229954717b2be14a299fb2612)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:22.532502 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b64573-394d-49db-a562-f92faff082cd admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:22.532788 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1059/2116] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:20:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:20:22.541996 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e4eec2-978e-49e8-8b6f-f047e6b63a78 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:22.542286 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e4eec2-978e-49e8-8b6f-f047e6b63a78 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:22.542619 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e4eec2-978e-49e8-8b6f-f047e6b63a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:22.563260 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-90e4eec2-978e-49e8-8b6f-f047e6b63a78 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:22.564489 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-90e4eec2-978e-49e8-8b6f-f047e6b63a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:22.573758 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1060/2117] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:20:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:20:22.578057 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e4eec2-978e-49e8-8b6f-f047e6b63a78 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:22.581584 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:22.584306 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-13b64573-394d-49db-a562-f92faff082cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7a9c31229954717b2be14a299fb2612'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:22.585153 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-13b64573-394d-49db-a562-f92faff082cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7a9c31229954717b2be14a299fb2612'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:22.588281 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e4eec2-978e-49e8-8b6f-f047e6b63a78 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:22.594046 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffffd7cb-a30a-40e7-aa4a-b2eeb5893069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:22.594389 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-13b64573-394d-49db-a562-f92faff082cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7a9c31229954717b2be14a299fb2612'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:22.594710 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffffd7cb-a30a-40e7-aa4a-b2eeb5893069 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:22.594995 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffffd7cb-a30a-40e7-aa4a-b2eeb5893069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:22.595286 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffffd7cb-a30a-40e7-aa4a-b2eeb5893069 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:22.596011 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffffd7cb-a30a-40e7-aa4a-b2eeb5893069 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:20:22.596319 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffffd7cb-a30a-40e7-aa4a-b2eeb5893069 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:22.597980 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffffd7cb-a30a-40e7-aa4a-b2eeb5893069 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:22.609849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-13b64573-394d-49db-a562-f92faff082cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7a9c31229954717b2be14a299fb2612'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:22.610563 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-13b64573-394d-49db-a562-f92faff082cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7a9c31229954717b2be14a299fb2612'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:22.613082 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1061/2118] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:20:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:20:22.615590 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffffd7cb-a30a-40e7-aa4a-b2eeb5893069 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:22.618785 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ffffd7cb-a30a-40e7-aa4a-b2eeb5893069 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:22.623860 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1058/2119] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:20:22 2022] DELETE /identity/v3/users/c7a9c31229954717b2be14a299fb2612 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:20:22.628088 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d91ca0a-c35c-48a0-9c2e-b86eb0486459 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:22.628386 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d91ca0a-c35c-48a0-9c2e-b86eb0486459 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:22.628686 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d91ca0a-c35c-48a0-9c2e-b86eb0486459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:22.632418 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffffd7cb-a30a-40e7-aa4a-b2eeb5893069 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:22.648147 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6d91ca0a-c35c-48a0-9c2e-b86eb0486459 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:22.649438 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6d91ca0a-c35c-48a0-9c2e-b86eb0486459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:22.673683 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:22.694983 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33c7479-2cef-4cae-9ff7-8b87459bda45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:22.695704 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c7479-2cef-4cae-9ff7-8b87459bda45 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:22.696000 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c7479-2cef-4cae-9ff7-8b87459bda45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:22.696279 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c7479-2cef-4cae-9ff7-8b87459bda45 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:22.696980 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33c7479-2cef-4cae-9ff7-8b87459bda45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:22.698926 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33c7479-2cef-4cae-9ff7-8b87459bda45 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:22.712313 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1059/2120] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:20:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:20:22.722754 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d91ca0a-c35c-48a0-9c2e-b86eb0486459 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:22.724085 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6d91ca0a-c35c-48a0-9c2e-b86eb0486459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:22.725852 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1062/2121] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:20:22 2022] POST /identity/v3/projects => generated 385 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:20:22.730096 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33c7479-2cef-4cae-9ff7-8b87459bda45 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:22.781184 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d91ca0a-c35c-48a0-9c2e-b86eb0486459 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:22.793344 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33c7479-2cef-4cae-9ff7-8b87459bda45 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:22.798047 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c158c318-db7f-428b-a5d0-4d08be05d0ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:22.798720 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158c318-db7f-428b-a5d0-4d08be05d0ac admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:22.799003 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158c318-db7f-428b-a5d0-4d08be05d0ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:22.799288 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158c318-db7f-428b-a5d0-4d08be05d0ac admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:22.800012 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c158c318-db7f-428b-a5d0-4d08be05d0ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:22.801979 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c158c318-db7f-428b-a5d0-4d08be05d0ac admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:22.822783 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1063/2122] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:20:22 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:20:22.826799 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c158c318-db7f-428b-a5d0-4d08be05d0ac admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:22.832872 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72540d4-9bdd-4479-89c0-be3e7a2f9b1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:22.833552 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72540d4-9bdd-4479-89c0-be3e7a2f9b1c service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:22.833834 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72540d4-9bdd-4479-89c0-be3e7a2f9b1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:22.834109 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72540d4-9bdd-4479-89c0-be3e7a2f9b1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:22.836576 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c158c318-db7f-428b-a5d0-4d08be05d0ac admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:22.841592 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c075a9-179b-4a92-9b1c-6763763ebfef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:22.842316 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c075a9-179b-4a92-9b1c-6763763ebfef admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:22.842600 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c075a9-179b-4a92-9b1c-6763763ebfef admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:22.842998 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c075a9-179b-4a92-9b1c-6763763ebfef admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:22.843733 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c075a9-179b-4a92-9b1c-6763763ebfef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:22.843838 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72540d4-9bdd-4479-89c0-be3e7a2f9b1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:22.845369 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c075a9-179b-4a92-9b1c-6763763ebfef admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:22.846575 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72540d4-9bdd-4479-89c0-be3e7a2f9b1c service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:22.856160 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1064/2123] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:20:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:20:22.861118 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c075a9-179b-4a92-9b1c-6763763ebfef admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:22.862619 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1060/2124] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:20:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:20:22.871760 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c075a9-179b-4a92-9b1c-6763763ebfef admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:22.876731 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd79dba9-642e-4558-add3-13c1ad754871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:22.877612 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd79dba9-642e-4558-add3-13c1ad754871 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:22.878023 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd79dba9-642e-4558-add3-13c1ad754871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:22.878479 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd79dba9-642e-4558-add3-13c1ad754871 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:22.879340 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd79dba9-642e-4558-add3-13c1ad754871 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:22.881017 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd79dba9-642e-4558-add3-13c1ad754871 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:22.895051 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1065/2125] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:20:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:20:22.896480 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72540d4-9bdd-4479-89c0-be3e7a2f9b1c service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:22.906775 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72540d4-9bdd-4479-89c0-be3e7a2f9b1c service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:22.911741 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8519003d-cd56-431a-aeb4-0f7ff22f6790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:22.912627 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519003d-cd56-431a-aeb4-0f7ff22f6790 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:22.912906 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519003d-cd56-431a-aeb4-0f7ff22f6790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:22.913215 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519003d-cd56-431a-aeb4-0f7ff22f6790 admin admin] PATH_INFO: `/v3/projects/04f1ffebada4489fb0e57498fd25c834/users/9e7bfbcf9ec64c6dac8de12f9e28c1f4/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:22.933605 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8519003d-cd56-431a-aeb4-0f7ff22f6790 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04f1ffebada4489fb0e57498fd25c834, user_id=9e7bfbcf9ec64c6dac8de12f9e28c1f4, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:22.936603 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8519003d-cd56-431a-aeb4-0f7ff22f6790 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:22.940264 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:22.952754 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1061/2126] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:20:22 2022] PUT /identity/v3/projects/04f1ffebada4489fb0e57498fd25c834/users/9e7bfbcf9ec64c6dac8de12f9e28c1f4/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:20:22.965695 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a6864-957b-473c-9bd3-a005a6b3a43c None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:22.965994 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a6864-957b-473c-9bd3-a005a6b3a43c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:22.966297 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a6864-957b-473c-9bd3-a005a6b3a43c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:22.995853 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-6a0a6864-957b-473c-9bd3-a005a6b3a43c None None] MFA Rules not processed for user `9e7bfbcf9ec64c6dac8de12f9e28c1f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:22.997127 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-6a0a6864-957b-473c-9bd3-a005a6b3a43c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:23.076600 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1066/2127] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:20:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:20:23.088686 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8519003d-cd56-431a-aeb4-0f7ff22f6790 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:23.090894 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8519003d-cd56-431a-aeb4-0f7ff22f6790 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:23.150751 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0a6864-957b-473c-9bd3-a005a6b3a43c None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:23.153154 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-6a0a6864-957b-473c-9bd3-a005a6b3a43c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:23.175955 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8519003d-cd56-431a-aeb4-0f7ff22f6790 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:23.180912 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b67326-7e7b-4691-9976-9650a43a89c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:23.181910 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b67326-7e7b-4691-9976-9650a43a89c3 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:23.182262 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b67326-7e7b-4691-9976-9650a43a89c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:23.182702 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b67326-7e7b-4691-9976-9650a43a89c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:23.198365 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:23.205527 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b67326-7e7b-4691-9976-9650a43a89c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:23.207069 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b67326-7e7b-4691-9976-9650a43a89c3 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:23.227889 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1062/2128] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:20:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:20:23.253061 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0a6864-957b-473c-9bd3-a005a6b3a43c None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:23.258602 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d80e203-01a7-4e5d-81c3-24d30d5dd712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:23.259618 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80e203-01a7-4e5d-81c3-24d30d5dd712 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:23.259912 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80e203-01a7-4e5d-81c3-24d30d5dd712 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:23.260192 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80e203-01a7-4e5d-81c3-24d30d5dd712 admin admin] PATH_INFO: `/v3/projects/c10c780b03114d81b50917c620f44de8` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:23.261431 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d80e203-01a7-4e5d-81c3-24d30d5dd712 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c10c780b03114d81b50917c620f44de8)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:23.263359 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d80e203-01a7-4e5d-81c3-24d30d5dd712 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:23.330220 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-6d80e203-01a7-4e5d-81c3-24d30d5dd712 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c10c780b03114d81b50917c620f44de8', 'request_id': 'req-6d80e203-01a7-4e5d-81c3-24d30d5dd712'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:23.339965 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-6d80e203-01a7-4e5d-81c3-24d30d5dd712 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c10c780b03114d81b50917c620f44de8', 'request_id': 'req-6d80e203-01a7-4e5d-81c3-24d30d5dd712'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:23.353685 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-6d80e203-01a7-4e5d-81c3-24d30d5dd712 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c10c780b03114d81b50917c620f44de8', 'request_id': 'req-6d80e203-01a7-4e5d-81c3-24d30d5dd712'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:23.378320 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-6d80e203-01a7-4e5d-81c3-24d30d5dd712 admin admin] The token cache is being invalidate because project c10c780b03114d81b50917c620f44de8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:20:23.378697 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-6d80e203-01a7-4e5d-81c3-24d30d5dd712 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:23.381590 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-6d80e203-01a7-4e5d-81c3-24d30d5dd712 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:23.385516 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1067/2129] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:20:23 2022] DELETE /identity/v3/projects/c10c780b03114d81b50917c620f44de8 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:20:23.389516 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b67326-7e7b-4691-9976-9650a43a89c3 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:23.412086 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b67326-7e7b-4691-9976-9650a43a89c3 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:23.500445 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d254381-b8f6-43ed-b9bd-3571d4fb604a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:23.501387 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d254381-b8f6-43ed-b9bd-3571d4fb604a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:23.501683 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d254381-b8f6-43ed-b9bd-3571d4fb604a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:23.502045 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d254381-b8f6-43ed-b9bd-3571d4fb604a admin admin] PATH_INFO: `/v3/users/4b1e677f4a93483d93ae754ecf8cfb16` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:23.517722 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d254381-b8f6-43ed-b9bd-3571d4fb604a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b1e677f4a93483d93ae754ecf8cfb16)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:23.522126 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d254381-b8f6-43ed-b9bd-3571d4fb604a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:23.615257 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4d254381-b8f6-43ed-b9bd-3571d4fb604a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b1e677f4a93483d93ae754ecf8cfb16'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:23.615933 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4d254381-b8f6-43ed-b9bd-3571d4fb604a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b1e677f4a93483d93ae754ecf8cfb16'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:23.625898 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4d254381-b8f6-43ed-b9bd-3571d4fb604a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b1e677f4a93483d93ae754ecf8cfb16'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:23.641686 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4d254381-b8f6-43ed-b9bd-3571d4fb604a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b1e677f4a93483d93ae754ecf8cfb16'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:23.642444 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4d254381-b8f6-43ed-b9bd-3571d4fb604a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b1e677f4a93483d93ae754ecf8cfb16'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:23.656025 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1063/2130] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:20:23 2022] DELETE /identity/v3/users/4b1e677f4a93483d93ae754ecf8cfb16 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:20:23.966713 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d80e203-01a7-4e5d-81c3-24d30d5dd712 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:23.968523 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-6d80e203-01a7-4e5d-81c3-24d30d5dd712 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:23.984336 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d80e203-01a7-4e5d-81c3-24d30d5dd712 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:24.046666 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f099c2-49b0-4c97-8943-2c3dd3d468d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:24.047536 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f099c2-49b0-4c97-8943-2c3dd3d468d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:24.047821 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f099c2-49b0-4c97-8943-2c3dd3d468d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:24.048102 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f099c2-49b0-4c97-8943-2c3dd3d468d4 admin admin] PATH_INFO: `/v3/projects/05da1a497c484ef68715eb405e664ee3` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:24.049358 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f099c2-49b0-4c97-8943-2c3dd3d468d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05da1a497c484ef68715eb405e664ee3)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:24.051321 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f099c2-49b0-4c97-8943-2c3dd3d468d4 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:24.098996 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c2f099c2-49b0-4c97-8943-2c3dd3d468d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05da1a497c484ef68715eb405e664ee3', 'request_id': 'req-c2f099c2-49b0-4c97-8943-2c3dd3d468d4'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:24.103493 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:24.109849 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c2f099c2-49b0-4c97-8943-2c3dd3d468d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05da1a497c484ef68715eb405e664ee3', 'request_id': 'req-c2f099c2-49b0-4c97-8943-2c3dd3d468d4'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:24.116891 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c2f099c2-49b0-4c97-8943-2c3dd3d468d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05da1a497c484ef68715eb405e664ee3', 'request_id': 'req-c2f099c2-49b0-4c97-8943-2c3dd3d468d4'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:24.130843 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c2f099c2-49b0-4c97-8943-2c3dd3d468d4 admin admin] The token cache is being invalidate because project 05da1a497c484ef68715eb405e664ee3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:20:24.131164 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c2f099c2-49b0-4c97-8943-2c3dd3d468d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:24.131805 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c2f099c2-49b0-4c97-8943-2c3dd3d468d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:24.133757 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1068/2131] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:20:23 2022] DELETE /identity/v3/projects/05da1a497c484ef68715eb405e664ee3 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:20:24.137696 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d254381-b8f6-43ed-b9bd-3571d4fb604a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:24.139189 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4d254381-b8f6-43ed-b9bd-3571d4fb604a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:24.152291 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d254381-b8f6-43ed-b9bd-3571d4fb604a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:24.167426 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:24.216380 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8c8df1-5060-4dfc-bd09-a93152bd8684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:24.217350 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c8df1-5060-4dfc-bd09-a93152bd8684 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:24.217644 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c8df1-5060-4dfc-bd09-a93152bd8684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:24.217929 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c8df1-5060-4dfc-bd09-a93152bd8684 admin admin] PATH_INFO: `/v3/users/128161c188d74caa98caa4822e9d2661` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:24.219131 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8c8df1-5060-4dfc-bd09-a93152bd8684 admin admin] RBAC: Authorizing `identity:delete_user(user_id=128161c188d74caa98caa4822e9d2661)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:24.221165 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8c8df1-5060-4dfc-bd09-a93152bd8684 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:24.272604 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ae8c8df1-5060-4dfc-bd09-a93152bd8684 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '128161c188d74caa98caa4822e9d2661'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:24.273296 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ae8c8df1-5060-4dfc-bd09-a93152bd8684 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '128161c188d74caa98caa4822e9d2661'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:24.299658 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ae8c8df1-5060-4dfc-bd09-a93152bd8684 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '128161c188d74caa98caa4822e9d2661'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:24.315630 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ae8c8df1-5060-4dfc-bd09-a93152bd8684 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '128161c188d74caa98caa4822e9d2661'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:24.316324 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ae8c8df1-5060-4dfc-bd09-a93152bd8684 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '128161c188d74caa98caa4822e9d2661'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:24.328807 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1064/2132] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:20:24 2022] DELETE /identity/v3/users/128161c188d74caa98caa4822e9d2661 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:20:24.672544 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f099c2-49b0-4c97-8943-2c3dd3d468d4 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:24.674171 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c2f099c2-49b0-4c97-8943-2c3dd3d468d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:24.692978 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f099c2-49b0-4c97-8943-2c3dd3d468d4 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:24.762420 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8c8df1-5060-4dfc-bd09-a93152bd8684 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:24.763942 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ae8c8df1-5060-4dfc-bd09-a93152bd8684 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:24.792323 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4febd2-c8f4-4025-ae65-2771ec182ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:24.794348 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4febd2-c8f4-4025-ae65-2771ec182ae3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:24.795288 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4febd2-c8f4-4025-ae65-2771ec182ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:24.795690 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4febd2-c8f4-4025-ae65-2771ec182ae3 admin admin] PATH_INFO: `/v3/projects/8359e756bdc94df7a7afc572bddde935` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:24.797270 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4febd2-c8f4-4025-ae65-2771ec182ae3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8359e756bdc94df7a7afc572bddde935)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:24.799462 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4febd2-c8f4-4025-ae65-2771ec182ae3 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:24.869647 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-1c4febd2-c8f4-4025-ae65-2771ec182ae3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8359e756bdc94df7a7afc572bddde935', 'request_id': 'req-1c4febd2-c8f4-4025-ae65-2771ec182ae3'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:24.874334 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8c8df1-5060-4dfc-bd09-a93152bd8684 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:24.878575 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-1c4febd2-c8f4-4025-ae65-2771ec182ae3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8359e756bdc94df7a7afc572bddde935', 'request_id': 'req-1c4febd2-c8f4-4025-ae65-2771ec182ae3'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:24.887656 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-1c4febd2-c8f4-4025-ae65-2771ec182ae3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8359e756bdc94df7a7afc572bddde935', 'request_id': 'req-1c4febd2-c8f4-4025-ae65-2771ec182ae3'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:24.906287 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-1c4febd2-c8f4-4025-ae65-2771ec182ae3 admin admin] The token cache is being invalidate because project 8359e756bdc94df7a7afc572bddde935 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:20:24.906658 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-1c4febd2-c8f4-4025-ae65-2771ec182ae3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:24.907528 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-1c4febd2-c8f4-4025-ae65-2771ec182ae3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:24.917099 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1069/2133] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:20:24 2022] DELETE /identity/v3/projects/8359e756bdc94df7a7afc572bddde935 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:20:24.969146 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:24.998408 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a820733b-2959-4b5a-b64b-fe47949ac961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:24.999898 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a820733b-2959-4b5a-b64b-fe47949ac961 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:25.000439 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a820733b-2959-4b5a-b64b-fe47949ac961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:25.000796 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a820733b-2959-4b5a-b64b-fe47949ac961 admin admin] PATH_INFO: `/v3/users/6db1042af2724042aa96119b5ce3147f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:25.003426 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a820733b-2959-4b5a-b64b-fe47949ac961 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6db1042af2724042aa96119b5ce3147f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:25.005624 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a820733b-2959-4b5a-b64b-fe47949ac961 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:25.063451 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a820733b-2959-4b5a-b64b-fe47949ac961 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6db1042af2724042aa96119b5ce3147f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:25.064288 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a820733b-2959-4b5a-b64b-fe47949ac961 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6db1042af2724042aa96119b5ce3147f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:25.075670 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a820733b-2959-4b5a-b64b-fe47949ac961 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6db1042af2724042aa96119b5ce3147f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:25.093443 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a820733b-2959-4b5a-b64b-fe47949ac961 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6db1042af2724042aa96119b5ce3147f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:25.094900 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a820733b-2959-4b5a-b64b-fe47949ac961 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6db1042af2724042aa96119b5ce3147f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:25.119084 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1065/2134] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:20:24 2022] DELETE /identity/v3/users/6db1042af2724042aa96119b5ce3147f => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:20:25.121824 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff83bd7-1279-4d6c-a1d1-a526d5b13571 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:25.122356 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff83bd7-1279-4d6c-a1d1-a526d5b13571 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:25.122779 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff83bd7-1279-4d6c-a1d1-a526d5b13571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:25.161068 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-5ff83bd7-1279-4d6c-a1d1-a526d5b13571 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:25.162716 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-5ff83bd7-1279-4d6c-a1d1-a526d5b13571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:25.229986 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1070/2135] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:20:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:20:25.240843 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a820733b-2959-4b5a-b64b-fe47949ac961 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:25.243068 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a820733b-2959-4b5a-b64b-fe47949ac961 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:25.306221 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a820733b-2959-4b5a-b64b-fe47949ac961 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:25.310373 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551d7473-f021-42c2-9094-3b90e2619d42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:25.310722 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d7473-f021-42c2-9094-3b90e2619d42 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:25.311016 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d7473-f021-42c2-9094-3b90e2619d42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:25.311297 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d7473-f021-42c2-9094-3b90e2619d42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:25.328755 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551d7473-f021-42c2-9094-3b90e2619d42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:25.330254 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551d7473-f021-42c2-9094-3b90e2619d42 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:25.342753 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1066/2136] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:20:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:20:25.787309 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff83bd7-1279-4d6c-a1d1-a526d5b13571 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:25.789204 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-5ff83bd7-1279-4d6c-a1d1-a526d5b13571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:25.802556 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff83bd7-1279-4d6c-a1d1-a526d5b13571 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:25.807819 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e4779-9030-4ac7-8c7d-e84884d3b994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:25.808770 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e4779-9030-4ac7-8c7d-e84884d3b994 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:25.809248 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e4779-9030-4ac7-8c7d-e84884d3b994 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:25.809533 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e4779-9030-4ac7-8c7d-e84884d3b994 admin admin] PATH_INFO: `/v3/projects/58c3e99e1a884a59966a4c18d934bbac` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:25.810970 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895e4779-9030-4ac7-8c7d-e84884d3b994 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58c3e99e1a884a59966a4c18d934bbac)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:25.812950 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895e4779-9030-4ac7-8c7d-e84884d3b994 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:25.823496 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:25.866055 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-895e4779-9030-4ac7-8c7d-e84884d3b994 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58c3e99e1a884a59966a4c18d934bbac', 'request_id': 'req-895e4779-9030-4ac7-8c7d-e84884d3b994'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:25.872083 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-895e4779-9030-4ac7-8c7d-e84884d3b994 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58c3e99e1a884a59966a4c18d934bbac', 'request_id': 'req-895e4779-9030-4ac7-8c7d-e84884d3b994'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:25.878541 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-895e4779-9030-4ac7-8c7d-e84884d3b994 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58c3e99e1a884a59966a4c18d934bbac', 'request_id': 'req-895e4779-9030-4ac7-8c7d-e84884d3b994'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:25.892420 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-895e4779-9030-4ac7-8c7d-e84884d3b994 admin admin] The token cache is being invalidate because project 58c3e99e1a884a59966a4c18d934bbac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:20:25.892717 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-895e4779-9030-4ac7-8c7d-e84884d3b994 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:25.893376 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-895e4779-9030-4ac7-8c7d-e84884d3b994 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:25.894932 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1071/2137] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:20:25 2022] DELETE /identity/v3/projects/58c3e99e1a884a59966a4c18d934bbac => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:20:25.901009 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ac71e-0ee8-4c65-aab7-83b8a7111989 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:25.901524 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ac71e-0ee8-4c65-aab7-83b8a7111989 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:25.901763 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ac71e-0ee8-4c65-aab7-83b8a7111989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:25.919374 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f49ac71e-0ee8-4c65-aab7-83b8a7111989 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:25.920611 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f49ac71e-0ee8-4c65-aab7-83b8a7111989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:25.997853 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1067/2138] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:20:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:20:26.006688 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0825cf10-2020-4c8b-b047-b547cb7e01b9 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:26.006985 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0825cf10-2020-4c8b-b047-b547cb7e01b9 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:26.007276 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0825cf10-2020-4c8b-b047-b547cb7e01b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:26.036568 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-0825cf10-2020-4c8b-b047-b547cb7e01b9 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:26.037851 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0825cf10-2020-4c8b-b047-b547cb7e01b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:26.047941 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1072/2139] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:20:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:20:26.051975 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ac71e-0ee8-4c65-aab7-83b8a7111989 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:26.063576 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ac71e-0ee8-4c65-aab7-83b8a7111989 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:26.070128 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38029d63-7265-44fa-859d-72ca4bb02aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:26.071895 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38029d63-7265-44fa-859d-72ca4bb02aeb admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:26.072187 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38029d63-7265-44fa-859d-72ca4bb02aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:26.072466 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38029d63-7265-44fa-859d-72ca4bb02aeb admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:26.073646 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38029d63-7265-44fa-859d-72ca4bb02aeb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:20:26.073941 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38029d63-7265-44fa-859d-72ca4bb02aeb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:26.075686 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38029d63-7265-44fa-859d-72ca4bb02aeb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:26.077415 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:26.093001 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1068/2140] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:20:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:20:26.097238 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0825cf10-2020-4c8b-b047-b547cb7e01b9 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:26.111297 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0825cf10-2020-4c8b-b047-b547cb7e01b9 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:26.124819 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f8dea1-6d3c-4813-8edc-c1d46dd97e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:26.125770 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f8dea1-6d3c-4813-8edc-c1d46dd97e4c admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:26.126189 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f8dea1-6d3c-4813-8edc-c1d46dd97e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:26.126794 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f8dea1-6d3c-4813-8edc-c1d46dd97e4c admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:26.127513 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f8dea1-6d3c-4813-8edc-c1d46dd97e4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:26.129471 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f8dea1-6d3c-4813-8edc-c1d46dd97e4c admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:26.160091 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1073/2141] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:20:26 2022] POST /identity/v3/projects => generated 383 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:20:26.161455 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38029d63-7265-44fa-859d-72ca4bb02aeb admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:26.173218 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38029d63-7265-44fa-859d-72ca4bb02aeb admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:26.230147 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b06486-fb89-450d-abce-b19181781ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:26.230762 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b06486-fb89-450d-abce-b19181781ea3 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:26.231049 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b06486-fb89-450d-abce-b19181781ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:26.231326 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b06486-fb89-450d-abce-b19181781ea3 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:26.232015 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b06486-fb89-450d-abce-b19181781ea3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:26.233980 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b06486-fb89-450d-abce-b19181781ea3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:26.257436 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1069/2142] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:20:26 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:20:26.259458 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f8dea1-6d3c-4813-8edc-c1d46dd97e4c admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:26.269888 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f8dea1-6d3c-4813-8edc-c1d46dd97e4c admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:26.275441 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f29e68-d5f9-482e-b51b-807592dfa0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:26.276103 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f29e68-d5f9-482e-b51b-807592dfa0d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:26.276408 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f29e68-d5f9-482e-b51b-807592dfa0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:26.276707 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f29e68-d5f9-482e-b51b-807592dfa0d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:26.277480 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f29e68-d5f9-482e-b51b-807592dfa0d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:26.279157 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f29e68-d5f9-482e-b51b-807592dfa0d9 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:26.290570 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1074/2143] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:20:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:20:26.294557 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b06486-fb89-450d-abce-b19181781ea3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:26.304602 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b06486-fb89-450d-abce-b19181781ea3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:26.309669 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f437d7-b6b7-4254-90a3-8a0f0e4b7794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:26.310333 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f437d7-b6b7-4254-90a3-8a0f0e4b7794 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:26.310621 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f437d7-b6b7-4254-90a3-8a0f0e4b7794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:26.310904 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f437d7-b6b7-4254-90a3-8a0f0e4b7794 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:26.311634 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f437d7-b6b7-4254-90a3-8a0f0e4b7794 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:26.313293 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f437d7-b6b7-4254-90a3-8a0f0e4b7794 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:26.327766 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1070/2144] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:20:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:20:26.332112 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f29e68-d5f9-482e-b51b-807592dfa0d9 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:26.348359 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f29e68-d5f9-482e-b51b-807592dfa0d9 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:26.354239 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5a3a3d-312f-42ae-8ad4-fe76f336804e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:26.355145 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a3a3d-312f-42ae-8ad4-fe76f336804e admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:26.355427 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a3a3d-312f-42ae-8ad4-fe76f336804e admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:26.355703 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a3a3d-312f-42ae-8ad4-fe76f336804e admin admin] PATH_INFO: `/v3/projects/96bc6b8b0235439587a3a5c9e837d708/users/65f1d0ab53de4473acee63b55bf14cab/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:26.371537 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5a3a3d-312f-42ae-8ad4-fe76f336804e admin admin] RBAC: Authorizing `identity:create_grant(project_id=96bc6b8b0235439587a3a5c9e837d708, user_id=65f1d0ab53de4473acee63b55bf14cab, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:26.374189 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5a3a3d-312f-42ae-8ad4-fe76f336804e admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:26.396305 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1075/2145] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:20:26 2022] PUT /identity/v3/projects/96bc6b8b0235439587a3a5c9e837d708/users/65f1d0ab53de4473acee63b55bf14cab/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:20:26.402007 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b3d1d8-9dcd-4f1b-9975-11b66de3a3ce None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:26.402287 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b3d1d8-9dcd-4f1b-9975-11b66de3a3ce None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:26.402563 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b3d1d8-9dcd-4f1b-9975-11b66de3a3ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:26.422134 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-48b3d1d8-9dcd-4f1b-9975-11b66de3a3ce None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:26.423354 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-48b3d1d8-9dcd-4f1b-9975-11b66de3a3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:26.514640 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1071/2146] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:20:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:20:26.524389 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5a3a3d-312f-42ae-8ad4-fe76f336804e admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:26.526981 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-5f5a3a3d-312f-42ae-8ad4-fe76f336804e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:26.596466 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5a3a3d-312f-42ae-8ad4-fe76f336804e admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:26.600517 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a8ff5c-e334-4e17-a811-766e811bdcd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:26.601273 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a8ff5c-e334-4e17-a811-766e811bdcd3 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:26.602136 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a8ff5c-e334-4e17-a811-766e811bdcd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:26.602551 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a8ff5c-e334-4e17-a811-766e811bdcd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:26.612563 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a8ff5c-e334-4e17-a811-766e811bdcd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:26.614125 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a8ff5c-e334-4e17-a811-766e811bdcd3 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:26.624575 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1076/2147] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:20:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:20:27.339536 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b3d1d8-9dcd-4f1b-9975-11b66de3a3ce None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:27.340870 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-48b3d1d8-9dcd-4f1b-9975-11b66de3a3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:27.355109 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b3d1d8-9dcd-4f1b-9975-11b66de3a3ce None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:27.361904 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c118118-9686-409d-9917-dcc9ee8421f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:27.363074 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c118118-9686-409d-9917-dcc9ee8421f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:27.363450 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c118118-9686-409d-9917-dcc9ee8421f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:27.363827 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c118118-9686-409d-9917-dcc9ee8421f7 admin admin] PATH_INFO: `/v3/users/e034114ab8244cc99dbf977b458a43f9` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:27.365386 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c118118-9686-409d-9917-dcc9ee8421f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e034114ab8244cc99dbf977b458a43f9)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:27.368127 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c118118-9686-409d-9917-dcc9ee8421f7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:27.430933 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd70cb-8385-4230-bd31-51e08a8584ba None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:27.431701 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd70cb-8385-4230-bd31-51e08a8584ba None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:27.431701 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd70cb-8385-4230-bd31-51e08a8584ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:27.444651 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8c118118-9686-409d-9917-dcc9ee8421f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e034114ab8244cc99dbf977b458a43f9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:27.446155 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8c118118-9686-409d-9917-dcc9ee8421f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e034114ab8244cc99dbf977b458a43f9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:27.454053 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8c118118-9686-409d-9917-dcc9ee8421f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e034114ab8244cc99dbf977b458a43f9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:27.467048 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-41cd70cb-8385-4230-bd31-51e08a8584ba None None] MFA Rules not processed for user `65f1d0ab53de4473acee63b55bf14cab`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:27.467225 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:27.468544 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-41cd70cb-8385-4230-bd31-51e08a8584ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:27.470223 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8c118118-9686-409d-9917-dcc9ee8421f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e034114ab8244cc99dbf977b458a43f9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:27.471242 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8c118118-9686-409d-9917-dcc9ee8421f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e034114ab8244cc99dbf977b458a43f9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:27.491416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1072/2148] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:20:27 2022] DELETE /identity/v3/users/e034114ab8244cc99dbf977b458a43f9 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:20:27.502714 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:27.550161 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1077/2149] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:20:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:20:27.562142 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c118118-9686-409d-9917-dcc9ee8421f7 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:27.563970 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8c118118-9686-409d-9917-dcc9ee8421f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:27.649070 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c118118-9686-409d-9917-dcc9ee8421f7 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:27.654037 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2686237-fcc7-4c20-9d28-77a324b4d193 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:27.654596 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2686237-fcc7-4c20-9d28-77a324b4d193 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:27.655408 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2686237-fcc7-4c20-9d28-77a324b4d193 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:27.655724 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2686237-fcc7-4c20-9d28-77a324b4d193 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:27.713501 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cd70cb-8385-4230-bd31-51e08a8584ba None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:27.713501 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-41cd70cb-8385-4230-bd31-51e08a8584ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:27.732296 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cd70cb-8385-4230-bd31-51e08a8584ba None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:27.734568 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2686237-fcc7-4c20-9d28-77a324b4d193 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:27.736115 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2686237-fcc7-4c20-9d28-77a324b4d193 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:27.747085 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1073/2150] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:20:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:20:27.800548 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26767862-fdd5-4e4a-9bc3-29dafc57f525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:27.801488 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26767862-fdd5-4e4a-9bc3-29dafc57f525 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:27.801778 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26767862-fdd5-4e4a-9bc3-29dafc57f525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:27.802057 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26767862-fdd5-4e4a-9bc3-29dafc57f525 admin admin] PATH_INFO: `/v3/projects/9acebc31809343e0abdc39ee5a127961` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:27.803280 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26767862-fdd5-4e4a-9bc3-29dafc57f525 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9acebc31809343e0abdc39ee5a127961)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:27.806022 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26767862-fdd5-4e4a-9bc3-29dafc57f525 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:27.854713 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-26767862-fdd5-4e4a-9bc3-29dafc57f525 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9acebc31809343e0abdc39ee5a127961', 'request_id': 'req-26767862-fdd5-4e4a-9bc3-29dafc57f525'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:27.860410 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-26767862-fdd5-4e4a-9bc3-29dafc57f525 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9acebc31809343e0abdc39ee5a127961', 'request_id': 'req-26767862-fdd5-4e4a-9bc3-29dafc57f525'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:27.866559 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-26767862-fdd5-4e4a-9bc3-29dafc57f525 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9acebc31809343e0abdc39ee5a127961', 'request_id': 'req-26767862-fdd5-4e4a-9bc3-29dafc57f525'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:27.879135 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-26767862-fdd5-4e4a-9bc3-29dafc57f525 admin admin] The token cache is being invalidate because project 9acebc31809343e0abdc39ee5a127961 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:20:27.879427 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-26767862-fdd5-4e4a-9bc3-29dafc57f525 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:27.880078 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-26767862-fdd5-4e4a-9bc3-29dafc57f525 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:27.881945 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1078/2151] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:20:27 2022] DELETE /identity/v3/projects/9acebc31809343e0abdc39ee5a127961 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:20:27.887187 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed3b92b-63fa-4075-9c46-87e079b1d1eb None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:27.887481 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed3b92b-63fa-4075-9c46-87e079b1d1eb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:27.888378 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed3b92b-63fa-4075-9c46-87e079b1d1eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:27.909140 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0ed3b92b-63fa-4075-9c46-87e079b1d1eb None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:27.910364 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0ed3b92b-63fa-4075-9c46-87e079b1d1eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:27.973149 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1074/2152] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:20:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:20:27.982573 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ccee9-e953-440d-ba99-a6c01b4011d2 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:27.982866 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ccee9-e953-440d-ba99-a6c01b4011d2 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:27.983151 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ccee9-e953-440d-ba99-a6c01b4011d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:28.003659 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-b56ccee9-e953-440d-ba99-a6c01b4011d2 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:28.005258 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b56ccee9-e953-440d-ba99-a6c01b4011d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:28.022090 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1079/2153] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:20:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:20:28.022274 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed3b92b-63fa-4075-9c46-87e079b1d1eb None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:28.032384 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed3b92b-63fa-4075-9c46-87e079b1d1eb None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:28.037150 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3cde66-3239-4786-8b03-f999e5078005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:28.037818 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3cde66-3239-4786-8b03-f999e5078005 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:28.038100 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3cde66-3239-4786-8b03-f999e5078005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:28.038378 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3cde66-3239-4786-8b03-f999e5078005 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:28.039639 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3cde66-3239-4786-8b03-f999e5078005 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:20:28.039950 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3cde66-3239-4786-8b03-f999e5078005 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:28.041575 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3cde66-3239-4786-8b03-f999e5078005 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:28.057801 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1075/2154] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:20:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:20:28.061973 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56ccee9-e953-440d-ba99-a6c01b4011d2 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:28.071510 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56ccee9-e953-440d-ba99-a6c01b4011d2 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:28.076761 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c2e313-fdbb-412a-b7ba-e9cc3be59ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:28.077598 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2e313-fdbb-412a-b7ba-e9cc3be59ec5 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:28.078022 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2e313-fdbb-412a-b7ba-e9cc3be59ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:28.078487 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2e313-fdbb-412a-b7ba-e9cc3be59ec5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:28.079327 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c2e313-fdbb-412a-b7ba-e9cc3be59ec5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:28.081598 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c2e313-fdbb-412a-b7ba-e9cc3be59ec5 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:28.108714 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1080/2155] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:20:28 2022] POST /identity/v3/projects => generated 389 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:20:28.112893 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3cde66-3239-4786-8b03-f999e5078005 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:28.124032 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3cde66-3239-4786-8b03-f999e5078005 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:28.153998 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:28.188915 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33e495a-f0d5-44a4-965f-96ca5c9c1ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:28.189691 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e495a-f0d5-44a4-965f-96ca5c9c1ee7 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:28.189979 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e495a-f0d5-44a4-965f-96ca5c9c1ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:28.190263 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e495a-f0d5-44a4-965f-96ca5c9c1ee7 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:28.190998 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33e495a-f0d5-44a4-965f-96ca5c9c1ee7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:28.192911 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33e495a-f0d5-44a4-965f-96ca5c9c1ee7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:28.212826 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1076/2156] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:20:28 2022] POST /identity/v3/users => generated 295 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:20:28.217435 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c2e313-fdbb-412a-b7ba-e9cc3be59ec5 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:28.227904 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c2e313-fdbb-412a-b7ba-e9cc3be59ec5 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:28.232687 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16abc110-5985-4f08-8451-07553f434c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:28.233379 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16abc110-5985-4f08-8451-07553f434c1d admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:28.233692 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16abc110-5985-4f08-8451-07553f434c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:28.233988 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16abc110-5985-4f08-8451-07553f434c1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:28.234714 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16abc110-5985-4f08-8451-07553f434c1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:28.236339 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16abc110-5985-4f08-8451-07553f434c1d admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:28.247655 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1081/2157] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:20:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:20:28.251587 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33e495a-f0d5-44a4-965f-96ca5c9c1ee7 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:28.262114 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33e495a-f0d5-44a4-965f-96ca5c9c1ee7 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:28.266783 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b76902a-8255-4f24-a6a2-1495ffaf33ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:28.267435 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76902a-8255-4f24-a6a2-1495ffaf33ee admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:28.267719 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76902a-8255-4f24-a6a2-1495ffaf33ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:28.268004 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76902a-8255-4f24-a6a2-1495ffaf33ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:28.268726 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b76902a-8255-4f24-a6a2-1495ffaf33ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:28.270358 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b76902a-8255-4f24-a6a2-1495ffaf33ee admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:28.280407 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1077/2158] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:20:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:20:28.284310 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16abc110-5985-4f08-8451-07553f434c1d admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:28.294594 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16abc110-5985-4f08-8451-07553f434c1d admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:28.299269 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c175edc-4c84-45a9-9564-5fd2e1449e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:28.300142 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c175edc-4c84-45a9-9564-5fd2e1449e71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:28.300420 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c175edc-4c84-45a9-9564-5fd2e1449e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:28.300695 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c175edc-4c84-45a9-9564-5fd2e1449e71 admin admin] PATH_INFO: `/v3/projects/930dc66606e94d9eb82cc50a67d009fb/users/9bcf554f3d6447399e1d9cfe08b0bff7/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:28.313200 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:28.316635 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c175edc-4c84-45a9-9564-5fd2e1449e71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=930dc66606e94d9eb82cc50a67d009fb, user_id=9bcf554f3d6447399e1d9cfe08b0bff7, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:28.319753 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c175edc-4c84-45a9-9564-5fd2e1449e71 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:28.335569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1082/2159] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:20:28 2022] PUT /identity/v3/projects/930dc66606e94d9eb82cc50a67d009fb/users/9bcf554f3d6447399e1d9cfe08b0bff7/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:20:28.341511 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee59a5e3-26f9-45b4-b3ff-fbca632641a6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:28.341981 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee59a5e3-26f9-45b4-b3ff-fbca632641a6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:28.342070 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee59a5e3-26f9-45b4-b3ff-fbca632641a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:28.359987 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ee59a5e3-26f9-45b4-b3ff-fbca632641a6 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:28.361233 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ee59a5e3-26f9-45b4-b3ff-fbca632641a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:28.425722 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1078/2160] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:20:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:20:28.436498 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c175edc-4c84-45a9-9564-5fd2e1449e71 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:28.438258 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-3c175edc-4c84-45a9-9564-5fd2e1449e71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:28.503029 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c175edc-4c84-45a9-9564-5fd2e1449e71 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:28.506625 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db67c20f-11cc-4c99-bc07-15f45e5b12a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:28.507269 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67c20f-11cc-4c99-bc07-15f45e5b12a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:28.507553 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67c20f-11cc-4c99-bc07-15f45e5b12a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:28.507894 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67c20f-11cc-4c99-bc07-15f45e5b12a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:28.517305 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db67c20f-11cc-4c99-bc07-15f45e5b12a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:28.518727 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db67c20f-11cc-4c99-bc07-15f45e5b12a1 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:28.528009 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1083/2161] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:20:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:20:29.116277 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f970b4b-4f3a-4c77-a295-870c26757067 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:29.116570 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f970b4b-4f3a-4c77-a295-870c26757067 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:29.116825 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f970b4b-4f3a-4c77-a295-870c26757067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:29.143524 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6f970b4b-4f3a-4c77-a295-870c26757067 None None] MFA Rules not processed for user `65f1d0ab53de4473acee63b55bf14cab`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:29.146775 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6f970b4b-4f3a-4c77-a295-870c26757067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:29.253112 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1079/2162] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:20:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:20:32.153338 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db67c20f-11cc-4c99-bc07-15f45e5b12a1 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:32.154850 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-db67c20f-11cc-4c99-bc07-15f45e5b12a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:32.173621 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db67c20f-11cc-4c99-bc07-15f45e5b12a1 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:32.183273 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682b8ae-5f0a-48dc-9f16-d3cd24e2099f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:32.186893 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682b8ae-5f0a-48dc-9f16-d3cd24e2099f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:32.187269 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682b8ae-5f0a-48dc-9f16-d3cd24e2099f admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:32.187651 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682b8ae-5f0a-48dc-9f16-d3cd24e2099f admin admin] PATH_INFO: `/v3/users/9e7bfbcf9ec64c6dac8de12f9e28c1f4` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:32.189385 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b682b8ae-5f0a-48dc-9f16-d3cd24e2099f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e7bfbcf9ec64c6dac8de12f9e28c1f4)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:32.191449 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b682b8ae-5f0a-48dc-9f16-d3cd24e2099f admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:32.271204 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b682b8ae-5f0a-48dc-9f16-d3cd24e2099f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e7bfbcf9ec64c6dac8de12f9e28c1f4'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:32.281163 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b682b8ae-5f0a-48dc-9f16-d3cd24e2099f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e7bfbcf9ec64c6dac8de12f9e28c1f4'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:32.282203 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b682b8ae-5f0a-48dc-9f16-d3cd24e2099f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e7bfbcf9ec64c6dac8de12f9e28c1f4'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:32.293286 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b682b8ae-5f0a-48dc-9f16-d3cd24e2099f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e7bfbcf9ec64c6dac8de12f9e28c1f4'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:32.331146 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b682b8ae-5f0a-48dc-9f16-d3cd24e2099f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e7bfbcf9ec64c6dac8de12f9e28c1f4'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:32.345092 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1084/2163] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:20:32 2022] DELETE /identity/v3/users/9e7bfbcf9ec64c6dac8de12f9e28c1f4 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:20:32.350805 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d57f1-af0c-46ee-8191-093667a8f82d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:32.353075 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d57f1-af0c-46ee-8191-093667a8f82d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:32.353647 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d57f1-af0c-46ee-8191-093667a8f82d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:32.383660 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4c6d57f1-af0c-46ee-8191-093667a8f82d None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:32.385704 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4c6d57f1-af0c-46ee-8191-093667a8f82d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:32.481805 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:32.527817 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1080/2164] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:20:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:20:32.538891 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682b8ae-5f0a-48dc-9f16-d3cd24e2099f admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:32.541122 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b682b8ae-5f0a-48dc-9f16-d3cd24e2099f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:32.634044 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682b8ae-5f0a-48dc-9f16-d3cd24e2099f admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:32.643969 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605d258e-5f0f-4f5b-87ef-5d4e850c2ccd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:32.644661 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605d258e-5f0f-4f5b-87ef-5d4e850c2ccd service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:32.644933 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605d258e-5f0f-4f5b-87ef-5d4e850c2ccd service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:32.645233 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605d258e-5f0f-4f5b-87ef-5d4e850c2ccd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:32.662743 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605d258e-5f0f-4f5b-87ef-5d4e850c2ccd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:32.664885 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605d258e-5f0f-4f5b-87ef-5d4e850c2ccd service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:32.678612 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1085/2165] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:20:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:20:33.798717 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6d57f1-af0c-46ee-8191-093667a8f82d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:33.800524 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4c6d57f1-af0c-46ee-8191-093667a8f82d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:33.813952 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2121ea-9337-4e54-a44f-4e600a787554 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:33.819392 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2121ea-9337-4e54-a44f-4e600a787554 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:33.819888 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2121ea-9337-4e54-a44f-4e600a787554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:33.833795 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6d57f1-af0c-46ee-8191-093667a8f82d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:33.846620 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e008ac49-a031-4ba6-8c3b-840fa4021a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:33.847577 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e008ac49-a031-4ba6-8c3b-840fa4021a7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:33.847870 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e008ac49-a031-4ba6-8c3b-840fa4021a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:33.848163 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e008ac49-a031-4ba6-8c3b-840fa4021a7e admin admin] PATH_INFO: `/v3/projects/04f1ffebada4489fb0e57498fd25c834` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:33.849402 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e008ac49-a031-4ba6-8c3b-840fa4021a7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=04f1ffebada4489fb0e57498fd25c834)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:33.850602 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:33.857124 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e008ac49-a031-4ba6-8c3b-840fa4021a7e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:33.864888 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-fe2121ea-9337-4e54-a44f-4e600a787554 None None] MFA Rules not processed for user `9bcf554f3d6447399e1d9cfe08b0bff7`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:33.866927 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-fe2121ea-9337-4e54-a44f-4e600a787554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:33.918707 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e008ac49-a031-4ba6-8c3b-840fa4021a7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04f1ffebada4489fb0e57498fd25c834', 'request_id': 'req-e008ac49-a031-4ba6-8c3b-840fa4021a7e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:33.924458 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e008ac49-a031-4ba6-8c3b-840fa4021a7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04f1ffebada4489fb0e57498fd25c834', 'request_id': 'req-e008ac49-a031-4ba6-8c3b-840fa4021a7e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:33.932561 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e008ac49-a031-4ba6-8c3b-840fa4021a7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04f1ffebada4489fb0e57498fd25c834', 'request_id': 'req-e008ac49-a031-4ba6-8c3b-840fa4021a7e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:33.949919 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e008ac49-a031-4ba6-8c3b-840fa4021a7e admin admin] The token cache is being invalidate because project 04f1ffebada4489fb0e57498fd25c834 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:20:33.950217 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e008ac49-a031-4ba6-8c3b-840fa4021a7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:33.951014 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e008ac49-a031-4ba6-8c3b-840fa4021a7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:33.954950 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1081/2166] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:20:33 2022] DELETE /identity/v3/projects/04f1ffebada4489fb0e57498fd25c834 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:20:33.962067 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9d7a96-e5a1-40e7-a95e-3da644d43aef None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:33.962859 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9d7a96-e5a1-40e7-a95e-3da644d43aef None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:33.962859 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9d7a96-e5a1-40e7-a95e-3da644d43aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:33.965358 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1086/2167] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:20:33 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:20:33.975588 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee6ccca-eb42-48d2-ab88-40aa6e3f9070 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:33.976146 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee6ccca-eb42-48d2-ab88-40aa6e3f9070 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:33.977024 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee6ccca-eb42-48d2-ab88-40aa6e3f9070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:33.985703 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-aa9d7a96-e5a1-40e7-a95e-3da644d43aef None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:33.986955 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-aa9d7a96-e5a1-40e7-a95e-3da644d43aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:33.997611 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-6ee6ccca-eb42-48d2-ab88-40aa6e3f9070 None None] MFA Rules not processed for user `9bcf554f3d6447399e1d9cfe08b0bff7`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:33.999526 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-6ee6ccca-eb42-48d2-ab88-40aa6e3f9070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:34.071962 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1082/2168] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:20:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:20:34.074449 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb34f1-23aa-497b-ba68-249deca01a97 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:34.074765 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb34f1-23aa-497b-ba68-249deca01a97 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:34.075049 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb34f1-23aa-497b-ba68-249deca01a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:34.096692 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1087/2169] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:20:33 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:20:34.104248 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee6ccca-eb42-48d2-ab88-40aa6e3f9070 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:34.105977 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-99cb34f1-23aa-497b-ba68-249deca01a97 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:34.106756 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-6ee6ccca-eb42-48d2-ab88-40aa6e3f9070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:34.107824 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-99cb34f1-23aa-497b-ba68-249deca01a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:34.117576 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1083/2170] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:20:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:20:34.121608 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb34f1-23aa-497b-ba68-249deca01a97 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:34.135852 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb34f1-23aa-497b-ba68-249deca01a97 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:34.140649 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdd4bad-535e-4433-9e73-8bf489f8eea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:34.141371 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdd4bad-535e-4433-9e73-8bf489f8eea5 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:34.141662 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdd4bad-535e-4433-9e73-8bf489f8eea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:34.141959 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdd4bad-535e-4433-9e73-8bf489f8eea5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:34.142698 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdd4bad-535e-4433-9e73-8bf489f8eea5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:20:34.143010 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdd4bad-535e-4433-9e73-8bf489f8eea5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:34.145176 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdd4bad-535e-4433-9e73-8bf489f8eea5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:34.165435 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1084/2171] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:20:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:20:34.169855 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdd4bad-535e-4433-9e73-8bf489f8eea5 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:34.188075 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdd4bad-535e-4433-9e73-8bf489f8eea5 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:34.193437 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e923235f-67a3-4783-a233-e10d119bbbf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:34.194315 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923235f-67a3-4783-a233-e10d119bbbf7 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:34.194609 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923235f-67a3-4783-a233-e10d119bbbf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:34.195003 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923235f-67a3-4783-a233-e10d119bbbf7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:34.196369 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e923235f-67a3-4783-a233-e10d119bbbf7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:34.198314 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e923235f-67a3-4783-a233-e10d119bbbf7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:34.241500 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee6ccca-eb42-48d2-ab88-40aa6e3f9070 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:34.243923 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1085/2172] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:20:34 2022] POST /identity/v3/projects => generated 375 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:20:34.255678 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e923235f-67a3-4783-a233-e10d119bbbf7 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:34.291169 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:34.299970 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:34.324751 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b910e8-c3fa-4572-91a6-7071457ec389 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15039da209ec445d93d49e2bd6d854eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:34.324751 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b910e8-c3fa-4572-91a6-7071457ec389 service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:34.324751 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b910e8-c3fa-4572-91a6-7071457ec389 service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:34.324751 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b910e8-c3fa-4572-91a6-7071457ec389 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:34.326984 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-86b910e8-c3fa-4572-91a6-7071457ec389 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:34.328962 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e923235f-67a3-4783-a233-e10d119bbbf7 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:34.333764 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baa4df2-4b3a-456e-b087-481b9e708095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:34.334472 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa4df2-4b3a-456e-b087-481b9e708095 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:34.334755 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa4df2-4b3a-456e-b087-481b9e708095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:34.335033 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa4df2-4b3a-456e-b087-481b9e708095 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:34.335755 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baa4df2-4b3a-456e-b087-481b9e708095 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:34.337864 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baa4df2-4b3a-456e-b087-481b9e708095 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:34.358394 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1086/2173] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:20:34 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:20:34.362561 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baa4df2-4b3a-456e-b087-481b9e708095 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:34.373439 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baa4df2-4b3a-456e-b087-481b9e708095 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:34.378258 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b44e8d7-4aea-45af-8f95-809cb00c553a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:34.379058 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b44e8d7-4aea-45af-8f95-809cb00c553a admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:34.379377 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b44e8d7-4aea-45af-8f95-809cb00c553a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:34.379661 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b44e8d7-4aea-45af-8f95-809cb00c553a admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:34.380476 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b44e8d7-4aea-45af-8f95-809cb00c553a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:34.382122 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b44e8d7-4aea-45af-8f95-809cb00c553a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:34.390714 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b910e8-c3fa-4572-91a6-7071457ec389 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:34.392361 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b910e8-c3fa-4572-91a6-7071457ec389 service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:34.396308 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1087/2174] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:20:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:20:34.397625 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b44e8d7-4aea-45af-8f95-809cb00c553a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:34.404717 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1088/2175] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:20:34 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:20:34.409318 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b44e8d7-4aea-45af-8f95-809cb00c553a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:34.413970 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4196c397-9a45-4c53-b995-8a3138591ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:34.415079 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4196c397-9a45-4c53-b995-8a3138591ec6 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:34.415388 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4196c397-9a45-4c53-b995-8a3138591ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:34.415696 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4196c397-9a45-4c53-b995-8a3138591ec6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:34.416359 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4196c397-9a45-4c53-b995-8a3138591ec6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:34.417982 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4196c397-9a45-4c53-b995-8a3138591ec6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:34.431781 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1088/2176] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:20:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:20:34.439038 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b910e8-c3fa-4572-91a6-7071457ec389 service nova] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:34.451876 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b910e8-c3fa-4572-91a6-7071457ec389 service nova] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:34.457356 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e1714-9c48-4617-8f10-fd36fd35a016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:34.458256 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e1714-9c48-4617-8f10-fd36fd35a016 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:34.458526 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e1714-9c48-4617-8f10-fd36fd35a016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:34.458808 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e1714-9c48-4617-8f10-fd36fd35a016 admin admin] PATH_INFO: `/v3/projects/bd31621c04cb404096fa811e987ab5df/users/071a72a4a8e4421786b9b401cd524a54/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:34.489701 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17e1714-9c48-4617-8f10-fd36fd35a016 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd31621c04cb404096fa811e987ab5df, user_id=071a72a4a8e4421786b9b401cd524a54, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:34.496718 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17e1714-9c48-4617-8f10-fd36fd35a016 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:34.515967 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1089/2177] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:20:34 2022] PUT /identity/v3/projects/bd31621c04cb404096fa811e987ab5df/users/071a72a4a8e4421786b9b401cd524a54/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:20:34.526387 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b4642-6ab4-4402-9a49-bb20fc888e28 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:34.526815 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b4642-6ab4-4402-9a49-bb20fc888e28 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:34.527234 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b4642-6ab4-4402-9a49-bb20fc888e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:34.559311 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3a0b4642-6ab4-4402-9a49-bb20fc888e28 None None] MFA Rules not processed for user `071a72a4a8e4421786b9b401cd524a54`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:34.560793 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3a0b4642-6ab4-4402-9a49-bb20fc888e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:34.647450 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1089/2178] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:20:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:20:34.648892 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e1714-9c48-4617-8f10-fd36fd35a016 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:34.748256 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e1714-9c48-4617-8f10-fd36fd35a016 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:34.754756 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85857d38-15e0-4b27-b71a-be81fb1d2c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:34.755155 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85857d38-15e0-4b27-b71a-be81fb1d2c39 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:34.755495 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85857d38-15e0-4b27-b71a-be81fb1d2c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:34.755787 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85857d38-15e0-4b27-b71a-be81fb1d2c39 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:34.756489 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85857d38-15e0-4b27-b71a-be81fb1d2c39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:34.759289 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85857d38-15e0-4b27-b71a-be81fb1d2c39 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:34.795233 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1090/2179] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:20:34 2022] POST /identity/v3/projects => generated 373 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:20:34.796397 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0b4642-6ab4-4402-9a49-bb20fc888e28 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:34.808985 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0b4642-6ab4-4402-9a49-bb20fc888e28 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:34.882121 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424af8d5-f0b3-434d-b188-9f2f86ffbe53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:34.883043 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424af8d5-f0b3-434d-b188-9f2f86ffbe53 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:34.883346 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424af8d5-f0b3-434d-b188-9f2f86ffbe53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:34.883917 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424af8d5-f0b3-434d-b188-9f2f86ffbe53 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:34.884673 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424af8d5-f0b3-434d-b188-9f2f86ffbe53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:34.886772 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424af8d5-f0b3-434d-b188-9f2f86ffbe53 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:34.909698 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1090/2180] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:20:34 2022] POST /identity/v3/users => generated 293 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:20:34.913600 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85857d38-15e0-4b27-b71a-be81fb1d2c39 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:34.925523 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85857d38-15e0-4b27-b71a-be81fb1d2c39 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:34.932417 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907f16ed-ee72-4eb7-8ec2-98efab0dde56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:34.933172 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f16ed-ee72-4eb7-8ec2-98efab0dde56 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:34.933465 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f16ed-ee72-4eb7-8ec2-98efab0dde56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:34.933746 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f16ed-ee72-4eb7-8ec2-98efab0dde56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:34.934485 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907f16ed-ee72-4eb7-8ec2-98efab0dde56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:34.936120 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907f16ed-ee72-4eb7-8ec2-98efab0dde56 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:34.948285 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1091/2181] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:20:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:20:34.952770 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424af8d5-f0b3-434d-b188-9f2f86ffbe53 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:34.966966 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424af8d5-f0b3-434d-b188-9f2f86ffbe53 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:34.971838 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082679b8-9d9a-4047-adae-1645343a5a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:34.973495 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082679b8-9d9a-4047-adae-1645343a5a46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:34.973936 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082679b8-9d9a-4047-adae-1645343a5a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:34.974344 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082679b8-9d9a-4047-adae-1645343a5a46 admin admin] PATH_INFO: `/v3/projects/a34a7df34f6148419eb0f2b4753ef84b/users/f9427be9a5354c2c8aa8aa6839daf83f/roles/f36e053b442540f08041ac7878760f65` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:34.994019 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082679b8-9d9a-4047-adae-1645343a5a46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a34a7df34f6148419eb0f2b4753ef84b, user_id=f9427be9a5354c2c8aa8aa6839daf83f, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:34.997355 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082679b8-9d9a-4047-adae-1645343a5a46 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:35.001475 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:35.017298 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1091/2182] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:20:34 2022] PUT /identity/v3/projects/a34a7df34f6148419eb0f2b4753ef84b/users/f9427be9a5354c2c8aa8aa6839daf83f/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:20:35.021142 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907f16ed-ee72-4eb7-8ec2-98efab0dde56 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:35.031971 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907f16ed-ee72-4eb7-8ec2-98efab0dde56 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:35.075717 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:35.122523 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd668f2-d11a-4ebe-90c9-71da536cbb0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:35.123579 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd668f2-d11a-4ebe-90c9-71da536cbb0f admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:35.124194 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd668f2-d11a-4ebe-90c9-71da536cbb0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:35.124926 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd668f2-d11a-4ebe-90c9-71da536cbb0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:35.125894 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd668f2-d11a-4ebe-90c9-71da536cbb0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:35.128550 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd668f2-d11a-4ebe-90c9-71da536cbb0f admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:35.154123 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082679b8-9d9a-4047-adae-1645343a5a46 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:35.157206 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1092/2183] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:20:35 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:20:35.158912 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc08e64-579e-4ec3-8e48-fb36fbfc67f7 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:35.159602 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc08e64-579e-4ec3-8e48-fb36fbfc67f7 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:35.160205 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc08e64-579e-4ec3-8e48-fb36fbfc67f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:35.168372 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082679b8-9d9a-4047-adae-1645343a5a46 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:35.176000 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b12d17-1d0d-44f4-930e-5c15e2fa8cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:35.177821 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b12d17-1d0d-44f4-930e-5c15e2fa8cee admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:35.178191 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b12d17-1d0d-44f4-930e-5c15e2fa8cee admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:35.178902 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b12d17-1d0d-44f4-930e-5c15e2fa8cee admin admin] PATH_INFO: `/v3/projects/a34a7df34f6148419eb0f2b4753ef84b/users/f9427be9a5354c2c8aa8aa6839daf83f/roles/f36e053b442540f08041ac7878760f65` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:35.183177 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b12d17-1d0d-44f4-930e-5c15e2fa8cee admin admin] RBAC: Authorizing `identity:create_grant(project_id=a34a7df34f6148419eb0f2b4753ef84b, user_id=f9427be9a5354c2c8aa8aa6839daf83f, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:35.183370 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-fdc08e64-579e-4ec3-8e48-fb36fbfc67f7 None None] MFA Rules not processed for user `9bcf554f3d6447399e1d9cfe08b0bff7`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:35.184733 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-fdc08e64-579e-4ec3-8e48-fb36fbfc67f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:35.188820 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b12d17-1d0d-44f4-930e-5c15e2fa8cee admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:35.209152 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1092/2184] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:20:35 2022] PUT /identity/v3/projects/a34a7df34f6148419eb0f2b4753ef84b/users/f9427be9a5354c2c8aa8aa6839daf83f/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:20:35.217356 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d299d1d-71f2-4446-b112-7022a38dd479 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:35.217646 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d299d1d-71f2-4446-b112-7022a38dd479 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:35.217952 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d299d1d-71f2-4446-b112-7022a38dd479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:35.261909 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5d299d1d-71f2-4446-b112-7022a38dd479 None None] MFA Rules not processed for user `f9427be9a5354c2c8aa8aa6839daf83f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:35.263397 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5d299d1d-71f2-4446-b112-7022a38dd479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:35.300851 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1093/2185] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:20:35 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:20:35.364740 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1093/2186] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:20:35 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:20:35.376773 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc08e64-579e-4ec3-8e48-fb36fbfc67f7 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:35.379305 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-fdc08e64-579e-4ec3-8e48-fb36fbfc67f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:35.527992 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc08e64-579e-4ec3-8e48-fb36fbfc67f7 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:35.660907 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c49eba4-87aa-4765-b4b7-ba9e948b3245 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:35.661615 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49eba4-87aa-4765-b4b7-ba9e948b3245 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:35.661910 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49eba4-87aa-4765-b4b7-ba9e948b3245 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:35.662533 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49eba4-87aa-4765-b4b7-ba9e948b3245 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:35.746114 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:35.774738 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c49eba4-87aa-4765-b4b7-ba9e948b3245 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:35.776397 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c49eba4-87aa-4765-b4b7-ba9e948b3245 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:35.791324 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1094/2187] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:20:35 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:20:38.613174 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d299d1d-71f2-4446-b112-7022a38dd479 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:38.732551 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d299d1d-71f2-4446-b112-7022a38dd479 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:38.737248 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e193816-cb60-4e45-88bd-18f5aa4d24f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:38.738126 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e193816-cb60-4e45-88bd-18f5aa4d24f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:38.738411 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e193816-cb60-4e45-88bd-18f5aa4d24f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:38.738689 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e193816-cb60-4e45-88bd-18f5aa4d24f6 admin admin] PATH_INFO: `/v3/users/071a72a4a8e4421786b9b401cd524a54` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:38.740102 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e193816-cb60-4e45-88bd-18f5aa4d24f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=071a72a4a8e4421786b9b401cd524a54)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:38.742639 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e193816-cb60-4e45-88bd-18f5aa4d24f6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:38.792714 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8e193816-cb60-4e45-88bd-18f5aa4d24f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '071a72a4a8e4421786b9b401cd524a54'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:38.793534 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8e193816-cb60-4e45-88bd-18f5aa4d24f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '071a72a4a8e4421786b9b401cd524a54'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:38.801920 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8e193816-cb60-4e45-88bd-18f5aa4d24f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '071a72a4a8e4421786b9b401cd524a54'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:38.813811 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:38.815194 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8e193816-cb60-4e45-88bd-18f5aa4d24f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '071a72a4a8e4421786b9b401cd524a54'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:38.816354 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8e193816-cb60-4e45-88bd-18f5aa4d24f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '071a72a4a8e4421786b9b401cd524a54'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:38.832453 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1094/2188] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:20:38 2022] DELETE /identity/v3/users/071a72a4a8e4421786b9b401cd524a54 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:20:38.837125 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df7de6a-bde9-4f9a-a127-217937f6cc3c None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:38.837664 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df7de6a-bde9-4f9a-a127-217937f6cc3c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:38.837809 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df7de6a-bde9-4f9a-a127-217937f6cc3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:38.857589 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-5df7de6a-bde9-4f9a-a127-217937f6cc3c None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:38.858834 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-5df7de6a-bde9-4f9a-a127-217937f6cc3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:38.944832 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1095/2189] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:20:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:20:38.955865 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e193816-cb60-4e45-88bd-18f5aa4d24f6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:38.959852 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8e193816-cb60-4e45-88bd-18f5aa4d24f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:39.070203 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e193816-cb60-4e45-88bd-18f5aa4d24f6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:39.074384 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67103eec-3093-4617-a2c3-72f6d8a8c1d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:39.075065 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67103eec-3093-4617-a2c3-72f6d8a8c1d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:39.075351 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67103eec-3093-4617-a2c3-72f6d8a8c1d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:39.075632 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67103eec-3093-4617-a2c3-72f6d8a8c1d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:39.086098 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67103eec-3093-4617-a2c3-72f6d8a8c1d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:39.088641 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67103eec-3093-4617-a2c3-72f6d8a8c1d3 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:39.112430 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1095/2190] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:20:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:20:39.581119 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df7de6a-bde9-4f9a-a127-217937f6cc3c None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:39.582646 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-5df7de6a-bde9-4f9a-a127-217937f6cc3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:39.597336 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df7de6a-bde9-4f9a-a127-217937f6cc3c None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:39.606797 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acb97c1-22b4-493f-a4ae-16be6699977d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:39.607703 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb97c1-22b4-493f-a4ae-16be6699977d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:39.607991 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb97c1-22b4-493f-a4ae-16be6699977d admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:39.608776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb97c1-22b4-493f-a4ae-16be6699977d admin admin] PATH_INFO: `/v3/projects/bd31621c04cb404096fa811e987ab5df` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:39.610106 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acb97c1-22b4-493f-a4ae-16be6699977d admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd31621c04cb404096fa811e987ab5df)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:39.612111 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acb97c1-22b4-493f-a4ae-16be6699977d admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:39.663590 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-6acb97c1-22b4-493f-a4ae-16be6699977d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd31621c04cb404096fa811e987ab5df', 'request_id': 'req-6acb97c1-22b4-493f-a4ae-16be6699977d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:39.671195 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-6acb97c1-22b4-493f-a4ae-16be6699977d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd31621c04cb404096fa811e987ab5df', 'request_id': 'req-6acb97c1-22b4-493f-a4ae-16be6699977d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:39.679017 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-6acb97c1-22b4-493f-a4ae-16be6699977d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd31621c04cb404096fa811e987ab5df', 'request_id': 'req-6acb97c1-22b4-493f-a4ae-16be6699977d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:39.692977 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-6acb97c1-22b4-493f-a4ae-16be6699977d admin admin] The token cache is being invalidate because project bd31621c04cb404096fa811e987ab5df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:20:39.693395 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-6acb97c1-22b4-493f-a4ae-16be6699977d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:39.694098 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-6acb97c1-22b4-493f-a4ae-16be6699977d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:39.696501 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1096/2191] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:20:39 2022] DELETE /identity/v3/projects/bd31621c04cb404096fa811e987ab5df => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:20:39.701270 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67103eec-3093-4617-a2c3-72f6d8a8c1d3 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:39.711519 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67103eec-3093-4617-a2c3-72f6d8a8c1d3 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:39.784190 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b346256-d2b7-40a2-948b-43369ad34be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:39.785415 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b346256-d2b7-40a2-948b-43369ad34be9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:39.785802 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b346256-d2b7-40a2-948b-43369ad34be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:39.786310 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b346256-d2b7-40a2-948b-43369ad34be9 admin admin] PATH_INFO: `/v3/users/f9427be9a5354c2c8aa8aa6839daf83f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:39.787911 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b346256-d2b7-40a2-948b-43369ad34be9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9427be9a5354c2c8aa8aa6839daf83f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:39.791508 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b346256-d2b7-40a2-948b-43369ad34be9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:39.860409 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6b346256-d2b7-40a2-948b-43369ad34be9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9427be9a5354c2c8aa8aa6839daf83f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:39.861291 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6b346256-d2b7-40a2-948b-43369ad34be9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9427be9a5354c2c8aa8aa6839daf83f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:39.873533 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6b346256-d2b7-40a2-948b-43369ad34be9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9427be9a5354c2c8aa8aa6839daf83f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:39.890659 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6b346256-d2b7-40a2-948b-43369ad34be9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9427be9a5354c2c8aa8aa6839daf83f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:39.891403 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6b346256-d2b7-40a2-948b-43369ad34be9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9427be9a5354c2c8aa8aa6839daf83f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:39.905076 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1096/2192] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:20:39 2022] DELETE /identity/v3/users/f9427be9a5354c2c8aa8aa6839daf83f => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:20:40.194343 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acb97c1-22b4-493f-a4ae-16be6699977d admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:40.196529 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-6acb97c1-22b4-493f-a4ae-16be6699977d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:40.213395 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acb97c1-22b4-493f-a4ae-16be6699977d admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:40.293889 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755cb9c7-9a33-4b97-875a-353bb57962c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:40.294887 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755cb9c7-9a33-4b97-875a-353bb57962c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:40.295247 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755cb9c7-9a33-4b97-875a-353bb57962c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:40.295618 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755cb9c7-9a33-4b97-875a-353bb57962c0 admin admin] PATH_INFO: `/v3/projects/a34a7df34f6148419eb0f2b4753ef84b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:40.296915 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755cb9c7-9a33-4b97-875a-353bb57962c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a34a7df34f6148419eb0f2b4753ef84b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:40.298864 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755cb9c7-9a33-4b97-875a-353bb57962c0 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:40.306779 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:40.360959 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-755cb9c7-9a33-4b97-875a-353bb57962c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a34a7df34f6148419eb0f2b4753ef84b', 'request_id': 'req-755cb9c7-9a33-4b97-875a-353bb57962c0'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:40.369208 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-755cb9c7-9a33-4b97-875a-353bb57962c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a34a7df34f6148419eb0f2b4753ef84b', 'request_id': 'req-755cb9c7-9a33-4b97-875a-353bb57962c0'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:40.376201 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-755cb9c7-9a33-4b97-875a-353bb57962c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a34a7df34f6148419eb0f2b4753ef84b', 'request_id': 'req-755cb9c7-9a33-4b97-875a-353bb57962c0'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:40.399073 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-755cb9c7-9a33-4b97-875a-353bb57962c0 admin admin] The token cache is being invalidate because project a34a7df34f6148419eb0f2b4753ef84b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:20:40.399404 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-755cb9c7-9a33-4b97-875a-353bb57962c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:40.400091 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-755cb9c7-9a33-4b97-875a-353bb57962c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:40.405282 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1097/2193] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:20:40 2022] DELETE /identity/v3/projects/a34a7df34f6148419eb0f2b4753ef84b => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:20:40.407023 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e12f44f-2347-4d38-9869-186e2de11602 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:40.407565 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e12f44f-2347-4d38-9869-186e2de11602 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:40.407783 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e12f44f-2347-4d38-9869-186e2de11602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:40.429093 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8e12f44f-2347-4d38-9869-186e2de11602 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:40.430978 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8e12f44f-2347-4d38-9869-186e2de11602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:40.437548 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:40.528974 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1097/2194] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:20:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:20:40.538648 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcadaf0-e490-4bcd-8e63-288a7b3f64e5 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:40.539536 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcadaf0-e490-4bcd-8e63-288a7b3f64e5 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:40.539867 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcadaf0-e490-4bcd-8e63-288a7b3f64e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:40.563789 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-7fcadaf0-e490-4bcd-8e63-288a7b3f64e5 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:40.565101 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7fcadaf0-e490-4bcd-8e63-288a7b3f64e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:40.583743 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1098/2195] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:20:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:20:40.584838 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e12f44f-2347-4d38-9869-186e2de11602 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:40.596201 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e12f44f-2347-4d38-9869-186e2de11602 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:40.608842 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb73e5-61aa-49c0-bf1f-ef67b80d063b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:40.609653 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb73e5-61aa-49c0-bf1f-ef67b80d063b admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:40.610014 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb73e5-61aa-49c0-bf1f-ef67b80d063b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:40.610415 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb73e5-61aa-49c0-bf1f-ef67b80d063b admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:40.611252 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfb73e5-61aa-49c0-bf1f-ef67b80d063b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:20:40.611637 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfb73e5-61aa-49c0-bf1f-ef67b80d063b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:40.614086 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfb73e5-61aa-49c0-bf1f-ef67b80d063b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:40.645533 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1098/2196] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:20:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:20:40.646900 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcadaf0-e490-4bcd-8e63-288a7b3f64e5 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:40.659121 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcadaf0-e490-4bcd-8e63-288a7b3f64e5 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:40.665228 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755c2f76-2126-480a-aa19-af704df5a13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:40.665751 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c2f76-2126-480a-aa19-af704df5a13d admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:40.666046 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c2f76-2126-480a-aa19-af704df5a13d admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:40.666332 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c2f76-2126-480a-aa19-af704df5a13d admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:40.667040 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755c2f76-2126-480a-aa19-af704df5a13d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:40.669009 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755c2f76-2126-480a-aa19-af704df5a13d admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:40.697494 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1099/2197] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:20:40 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:20:40.701805 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb73e5-61aa-49c0-bf1f-ef67b80d063b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:40.712784 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb73e5-61aa-49c0-bf1f-ef67b80d063b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:40.776630 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1a423b-f4b9-4aa6-8c6f-abb4aea5ba67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:40.777344 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a423b-f4b9-4aa6-8c6f-abb4aea5ba67 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:40.777656 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a423b-f4b9-4aa6-8c6f-abb4aea5ba67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:40.777939 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a423b-f4b9-4aa6-8c6f-abb4aea5ba67 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:40.778644 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1a423b-f4b9-4aa6-8c6f-abb4aea5ba67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:40.780555 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1a423b-f4b9-4aa6-8c6f-abb4aea5ba67 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:40.800943 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1099/2198] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:20:40 2022] POST /identity/v3/users => generated 286 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:20:40.805961 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755c2f76-2126-480a-aa19-af704df5a13d admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:40.816361 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755c2f76-2126-480a-aa19-af704df5a13d admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:40.821110 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8813d1a-0af7-4664-bf4f-fa644d56dab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:40.821771 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8813d1a-0af7-4664-bf4f-fa644d56dab7 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:40.822061 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8813d1a-0af7-4664-bf4f-fa644d56dab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:40.822342 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8813d1a-0af7-4664-bf4f-fa644d56dab7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:40.823068 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8813d1a-0af7-4664-bf4f-fa644d56dab7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:40.824673 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8813d1a-0af7-4664-bf4f-fa644d56dab7 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:40.836576 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1100/2199] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:20:40.840666 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1a423b-f4b9-4aa6-8c6f-abb4aea5ba67 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:40.853285 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1a423b-f4b9-4aa6-8c6f-abb4aea5ba67 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:40.858065 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26db415-6f3e-47e2-af41-b6a9736b86b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:40.858681 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26db415-6f3e-47e2-af41-b6a9736b86b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:40.858971 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26db415-6f3e-47e2-af41-b6a9736b86b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:40.859252 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26db415-6f3e-47e2-af41-b6a9736b86b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:40.859982 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26db415-6f3e-47e2-af41-b6a9736b86b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:40.861606 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26db415-6f3e-47e2-af41-b6a9736b86b9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:40.872564 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1100/2200] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:20:40.876945 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8813d1a-0af7-4664-bf4f-fa644d56dab7 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:40.887188 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8813d1a-0af7-4664-bf4f-fa644d56dab7 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:40.891888 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaf1ec6-456c-43c2-b30b-3ab94bc88edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:40.892768 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaf1ec6-456c-43c2-b30b-3ab94bc88edb admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:40.893080 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaf1ec6-456c-43c2-b30b-3ab94bc88edb admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:40.893394 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaf1ec6-456c-43c2-b30b-3ab94bc88edb admin admin] PATH_INFO: `/v3/projects/d6ecddd48b9a466880e0288edd65ce0a/users/44e20abe50c041c0b52d788b1cb237fe/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:40.908251 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaf1ec6-456c-43c2-b30b-3ab94bc88edb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6ecddd48b9a466880e0288edd65ce0a, user_id=44e20abe50c041c0b52d788b1cb237fe, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:40.910961 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaf1ec6-456c-43c2-b30b-3ab94bc88edb admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:40.925059 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1101/2201] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:20:40 2022] PUT /identity/v3/projects/d6ecddd48b9a466880e0288edd65ce0a/users/44e20abe50c041c0b52d788b1cb237fe/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:20:40.930256 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff808517-4b92-46b6-9721-b0ca103940ce None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:40.930935 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff808517-4b92-46b6-9721-b0ca103940ce None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:40.930935 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff808517-4b92-46b6-9721-b0ca103940ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:40.948793 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ff808517-4b92-46b6-9721-b0ca103940ce None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:40.950045 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ff808517-4b92-46b6-9721-b0ca103940ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:41.012354 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1101/2202] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:20:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:20:41.021962 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaf1ec6-456c-43c2-b30b-3ab94bc88edb admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:41.024576 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7eaf1ec6-456c-43c2-b30b-3ab94bc88edb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:41.083614 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaf1ec6-456c-43c2-b30b-3ab94bc88edb admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:41.088167 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc81d9b-46e2-4a9e-845d-ec2ed7ebffcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:41.088827 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc81d9b-46e2-4a9e-845d-ec2ed7ebffcf service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:41.089458 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc81d9b-46e2-4a9e-845d-ec2ed7ebffcf service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:41.090135 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc81d9b-46e2-4a9e-845d-ec2ed7ebffcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:41.099402 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc81d9b-46e2-4a9e-845d-ec2ed7ebffcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:41.100846 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc81d9b-46e2-4a9e-845d-ec2ed7ebffcf service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:41.111569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1102/2203] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:20:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:20:41.820076 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4327dc5d-acc7-473b-b96c-261be99642b4 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:41.821218 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4327dc5d-acc7-473b-b96c-261be99642b4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:41.821625 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4327dc5d-acc7-473b-b96c-261be99642b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:41.866395 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4327dc5d-acc7-473b-b96c-261be99642b4 None None] MFA Rules not processed for user `44e20abe50c041c0b52d788b1cb237fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:41.867789 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4327dc5d-acc7-473b-b96c-261be99642b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:41.872367 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:41.937641 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1102/2204] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:20:41 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:20:41.949234 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc81d9b-46e2-4a9e-845d-ec2ed7ebffcf service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:41.950732 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-5cc81d9b-46e2-4a9e-845d-ec2ed7ebffcf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:41.988672 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:42.016847 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc81d9b-46e2-4a9e-845d-ec2ed7ebffcf service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:42.020504 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cef1ede-64a9-416d-8839-392b0520a88b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:42.021242 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef1ede-64a9-416d-8839-392b0520a88b service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:42.021535 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef1ede-64a9-416d-8839-392b0520a88b service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:42.021819 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef1ede-64a9-416d-8839-392b0520a88b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:42.030356 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cef1ede-64a9-416d-8839-392b0520a88b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:42.031795 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cef1ede-64a9-416d-8839-392b0520a88b service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:42.041321 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1103/2205] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:20:41 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:20:43.579884 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9625836-93bb-4bde-9cf1-0b27a40dfa10 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:43.580734 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9625836-93bb-4bde-9cf1-0b27a40dfa10 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:43.581228 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9625836-93bb-4bde-9cf1-0b27a40dfa10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:43.624294 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c9625836-93bb-4bde-9cf1-0b27a40dfa10 None None] MFA Rules not processed for user `44e20abe50c041c0b52d788b1cb237fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:43.630353 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c9625836-93bb-4bde-9cf1-0b27a40dfa10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:43.654203 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1103/2206] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:20:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:20:52.036614 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=11 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:53.606216 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=28 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:55.250076 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524418dd-b073-43f1-b94c-1a89d02e5328 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:55.250570 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524418dd-b073-43f1-b94c-1a89d02e5328 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:55.250659 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524418dd-b073-43f1-b94c-1a89d02e5328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:55.268843 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-524418dd-b073-43f1-b94c-1a89d02e5328 None None] MFA Rules not processed for user `6bc30b7d536444008e5b5bab45c5c438`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:55.270097 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-524418dd-b073-43f1-b94c-1a89d02e5328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:55.333301 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1104/2207] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:20:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:21:03.063067 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa752641-5798-4443-a1bc-dfdad1e0e849 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:03.063714 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa752641-5798-4443-a1bc-dfdad1e0e849 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:03.063714 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa752641-5798-4443-a1bc-dfdad1e0e849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:03.083972 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-fa752641-5798-4443-a1bc-dfdad1e0e849 None None] MFA Rules not processed for user `44e20abe50c041c0b52d788b1cb237fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:21:03.085293 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fa752641-5798-4443-a1bc-dfdad1e0e849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:03.093859 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1104/2208] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:21:03 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:21:05.326576 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=25 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:21:13.078544 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:21:33.114225 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23ea156-671b-4375-80c2-4b2acb09a7f5 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:33.114723 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23ea156-671b-4375-80c2-4b2acb09a7f5 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:33.114827 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23ea156-671b-4375-80c2-4b2acb09a7f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:33.133482 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-f23ea156-671b-4375-80c2-4b2acb09a7f5 None None] MFA Rules not processed for user `78668b939c5847b1b73e9c28b5d8db7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:21:33.136708 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f23ea156-671b-4375-80c2-4b2acb09a7f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:33.195169 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1105/2209] 10.222.0.71 () {58 vars in 1009 bytes} [Mon Jul 4 09:21:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:21:33.242804 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa752641-5798-4443-a1bc-dfdad1e0e849 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:33.244228 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fa752641-5798-4443-a1bc-dfdad1e0e849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:33.324966 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa752641-5798-4443-a1bc-dfdad1e0e849 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:33.330173 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d416d5a9-4e40-43b0-a3a7-9bffc1c46925 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88b41b55769a4c449716c3590ed58225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:33.330831 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d416d5a9-4e40-43b0-a3a7-9bffc1c46925 service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:33.331120 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d416d5a9-4e40-43b0-a3a7-9bffc1c46925 service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:33.331401 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d416d5a9-4e40-43b0-a3a7-9bffc1c46925 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:33.340324 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d416d5a9-4e40-43b0-a3a7-9bffc1c46925 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:33.342325 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d416d5a9-4e40-43b0-a3a7-9bffc1c46925 service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:33.350634 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1105/2210] 10.222.0.71 () {60 vars in 1384 bytes} [Mon Jul 4 09:21:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 04 09:21:33.537411 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c934e8df-2d02-43d2-9118-d43410da26fb None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:33.537928 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c934e8df-2d02-43d2-9118-d43410da26fb None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:33.538468 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c934e8df-2d02-43d2-9118-d43410da26fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:33.564417 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-c934e8df-2d02-43d2-9118-d43410da26fb None None] MFA Rules not processed for user `78668b939c5847b1b73e9c28b5d8db7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:21:33.565700 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c934e8df-2d02-43d2-9118-d43410da26fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:33.581262 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1106/2211] 10.222.0.71 () {58 vars in 1009 bytes} [Mon Jul 4 09:21:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:21:33.610664 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d416d5a9-4e40-43b0-a3a7-9bffc1c46925 service swift] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:33.630829 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d416d5a9-4e40-43b0-a3a7-9bffc1c46925 service swift] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:33.641461 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cff9283-4f71-43b3-b7cd-b13b3f80e5ac service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88b41b55769a4c449716c3590ed58225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:33.642125 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff9283-4f71-43b3-b7cd-b13b3f80e5ac service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:33.642789 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff9283-4f71-43b3-b7cd-b13b3f80e5ac service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:33.643071 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff9283-4f71-43b3-b7cd-b13b3f80e5ac service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:33.668725 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cff9283-4f71-43b3-b7cd-b13b3f80e5ac service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:33.678095 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cff9283-4f71-43b3-b7cd-b13b3f80e5ac service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:33.693550 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1106/2212] 10.222.0.71 () {60 vars in 1384 bytes} [Mon Jul 4 09:21:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 09:21:43.556169 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=28 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:21:43.689078 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=38 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:21:49.411622 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839f85eb-1b21-4848-b262-44861c97ab55 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:49.412137 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839f85eb-1b21-4848-b262-44861c97ab55 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:49.412219 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839f85eb-1b21-4848-b262-44861c97ab55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:49.463021 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-839f85eb-1b21-4848-b262-44861c97ab55 None None] MFA Rules not processed for user `65f1d0ab53de4473acee63b55bf14cab`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:21:49.465939 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-839f85eb-1b21-4848-b262-44861c97ab55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:49.584620 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1107/2213] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:21:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:21:51.051932 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b7e391-47bf-4ac4-a5d2-12230ff45e0b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:51.052419 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b7e391-47bf-4ac4-a5d2-12230ff45e0b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:51.052510 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b7e391-47bf-4ac4-a5d2-12230ff45e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:51.080480 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-15b7e391-47bf-4ac4-a5d2-12230ff45e0b None None] MFA Rules not processed for user `78668b939c5847b1b73e9c28b5d8db7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:21:51.081733 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-15b7e391-47bf-4ac4-a5d2-12230ff45e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:51.105820 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1107/2214] 10.222.0.71 () {58 vars in 967 bytes} [Mon Jul 4 09:21:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:21:51.118068 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839f85eb-1b21-4848-b262-44861c97ab55 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:51.137686 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839f85eb-1b21-4848-b262-44861c97ab55 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:51.142511 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03e6419-b4e9-430b-bb02-5b8d2b45bf5a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88b41b55769a4c449716c3590ed58225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:51.143176 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e6419-b4e9-430b-bb02-5b8d2b45bf5a service swift] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:51.143463 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e6419-b4e9-430b-bb02-5b8d2b45bf5a service swift] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:51.143741 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e6419-b4e9-430b-bb02-5b8d2b45bf5a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:51.151813 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03e6419-b4e9-430b-bb02-5b8d2b45bf5a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:51.153458 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03e6419-b4e9-430b-bb02-5b8d2b45bf5a service swift] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:51.161644 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1108/2215] 10.222.0.71 () {60 vars in 1384 bytes} [Mon Jul 4 09:21:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 09:21:53.202335 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b7e391-47bf-4ac4-a5d2-12230ff45e0b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:53.233336 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b7e391-47bf-4ac4-a5d2-12230ff45e0b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:53.237889 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c158036-94b1-4287-afb3-5b334c44feab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:53.238747 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c158036-94b1-4287-afb3-5b334c44feab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:53.239032 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c158036-94b1-4287-afb3-5b334c44feab admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:53.239311 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c158036-94b1-4287-afb3-5b334c44feab admin admin] PATH_INFO: `/v3/users/44e20abe50c041c0b52d788b1cb237fe` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:53.240502 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c158036-94b1-4287-afb3-5b334c44feab admin admin] RBAC: Authorizing `identity:delete_user(user_id=44e20abe50c041c0b52d788b1cb237fe)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:53.242431 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c158036-94b1-4287-afb3-5b334c44feab admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:53.288878 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5c158036-94b1-4287-afb3-5b334c44feab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44e20abe50c041c0b52d788b1cb237fe'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:53.289568 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5c158036-94b1-4287-afb3-5b334c44feab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44e20abe50c041c0b52d788b1cb237fe'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:53.296961 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5c158036-94b1-4287-afb3-5b334c44feab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44e20abe50c041c0b52d788b1cb237fe'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:53.309176 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5c158036-94b1-4287-afb3-5b334c44feab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44e20abe50c041c0b52d788b1cb237fe'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:53.309811 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5c158036-94b1-4287-afb3-5b334c44feab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44e20abe50c041c0b52d788b1cb237fe'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:53.321580 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1108/2216] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:21:53 2022] DELETE /identity/v3/users/44e20abe50c041c0b52d788b1cb237fe => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:21:53.498652 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03e6419-b4e9-430b-bb02-5b8d2b45bf5a service swift] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:53.509844 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03e6419-b4e9-430b-bb02-5b8d2b45bf5a service swift] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:53.590185 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f55edfd-130a-45ef-9302-78411c193473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:53.591072 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55edfd-130a-45ef-9302-78411c193473 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:53.591358 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55edfd-130a-45ef-9302-78411c193473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:53.591638 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55edfd-130a-45ef-9302-78411c193473 admin admin] PATH_INFO: `/v3/projects/d6ecddd48b9a466880e0288edd65ce0a` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:53.592866 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f55edfd-130a-45ef-9302-78411c193473 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6ecddd48b9a466880e0288edd65ce0a)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:53.594825 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f55edfd-130a-45ef-9302-78411c193473 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:53.647092 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-2f55edfd-130a-45ef-9302-78411c193473 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6ecddd48b9a466880e0288edd65ce0a', 'request_id': 'req-2f55edfd-130a-45ef-9302-78411c193473'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:53.652878 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-2f55edfd-130a-45ef-9302-78411c193473 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6ecddd48b9a466880e0288edd65ce0a', 'request_id': 'req-2f55edfd-130a-45ef-9302-78411c193473'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:53.658013 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:21:53.663460 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-2f55edfd-130a-45ef-9302-78411c193473 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6ecddd48b9a466880e0288edd65ce0a', 'request_id': 'req-2f55edfd-130a-45ef-9302-78411c193473'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:53.689976 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-2f55edfd-130a-45ef-9302-78411c193473 admin admin] The token cache is being invalidate because project d6ecddd48b9a466880e0288edd65ce0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:21:53.690267 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-2f55edfd-130a-45ef-9302-78411c193473 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:53.690908 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-2f55edfd-130a-45ef-9302-78411c193473 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:53.692471 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1109/2217] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:21:53 2022] DELETE /identity/v3/projects/d6ecddd48b9a466880e0288edd65ce0a => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:21:53.697478 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a919e9-4bc5-43ea-ad84-d97dbd0ececd None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:53.697796 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a919e9-4bc5-43ea-ad84-d97dbd0ececd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:53.698082 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a919e9-4bc5-43ea-ad84-d97dbd0ececd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:53.761608 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-83a919e9-4bc5-43ea-ad84-d97dbd0ececd None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:21:53.763056 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-83a919e9-4bc5-43ea-ad84-d97dbd0ececd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:53.815901 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:21:53.834856 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1109/2218] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:21:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:21:53.845774 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22444974-8bd4-476b-b147-c646e5a53531 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:53.846067 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22444974-8bd4-476b-b147-c646e5a53531 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:53.846349 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22444974-8bd4-476b-b147-c646e5a53531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:53.864243 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-22444974-8bd4-476b-b147-c646e5a53531 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:21:53.865493 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-22444974-8bd4-476b-b147-c646e5a53531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:53.874338 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1110/2219] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:21:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:21:53.878715 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a919e9-4bc5-43ea-ad84-d97dbd0ececd None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:53.890828 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a919e9-4bc5-43ea-ad84-d97dbd0ececd None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:53.895902 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b3190a-703c-4157-b43c-87d2ac27dae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:53.896670 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b3190a-703c-4157-b43c-87d2ac27dae7 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:53.896956 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b3190a-703c-4157-b43c-87d2ac27dae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:53.897277 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b3190a-703c-4157-b43c-87d2ac27dae7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:53.898111 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b3190a-703c-4157-b43c-87d2ac27dae7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:21:53.898459 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b3190a-703c-4157-b43c-87d2ac27dae7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:53.900111 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b3190a-703c-4157-b43c-87d2ac27dae7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:53.912628 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1110/2220] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:21:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:21:53.917809 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22444974-8bd4-476b-b147-c646e5a53531 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:53.928117 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22444974-8bd4-476b-b147-c646e5a53531 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:53.933796 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39da6e70-f164-4915-98e7-5a5a1c0ddcc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:53.934552 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da6e70-f164-4915-98e7-5a5a1c0ddcc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:53.934883 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da6e70-f164-4915-98e7-5a5a1c0ddcc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:53.935163 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da6e70-f164-4915-98e7-5a5a1c0ddcc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:53.935884 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39da6e70-f164-4915-98e7-5a5a1c0ddcc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:53.938137 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39da6e70-f164-4915-98e7-5a5a1c0ddcc1 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:53.968702 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1111/2221] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:21:53 2022] POST /identity/v3/projects => generated 369 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:21:53.973183 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b3190a-703c-4157-b43c-87d2ac27dae7 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:53.984517 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b3190a-703c-4157-b43c-87d2ac27dae7 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:54.050670 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aa1525-f3c4-43cd-9df9-7c0e378e0807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:54.051503 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa1525-f3c4-43cd-9df9-7c0e378e0807 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:54.051865 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa1525-f3c4-43cd-9df9-7c0e378e0807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:54.052247 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa1525-f3c4-43cd-9df9-7c0e378e0807 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:54.053023 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aa1525-f3c4-43cd-9df9-7c0e378e0807 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:54.055149 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aa1525-f3c4-43cd-9df9-7c0e378e0807 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:54.082312 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1111/2222] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:21:53 2022] POST /identity/v3/users => generated 285 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:21:54.086103 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39da6e70-f164-4915-98e7-5a5a1c0ddcc1 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:54.097858 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39da6e70-f164-4915-98e7-5a5a1c0ddcc1 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:54.104291 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b8ad4c-38ad-4c8b-a011-21ed3d134d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:54.105591 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b8ad4c-38ad-4c8b-a011-21ed3d134d6f admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:54.105881 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b8ad4c-38ad-4c8b-a011-21ed3d134d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:54.106310 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b8ad4c-38ad-4c8b-a011-21ed3d134d6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:54.106983 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b8ad4c-38ad-4c8b-a011-21ed3d134d6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:54.110006 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b8ad4c-38ad-4c8b-a011-21ed3d134d6f admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:54.124712 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1112/2223] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:21:54 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:21:54.129393 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aa1525-f3c4-43cd-9df9-7c0e378e0807 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:54.151298 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aa1525-f3c4-43cd-9df9-7c0e378e0807 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:54.167788 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e498b069-8f67-4255-9008-774408fd1b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:54.168531 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e498b069-8f67-4255-9008-774408fd1b8c admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:54.169093 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e498b069-8f67-4255-9008-774408fd1b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:54.169436 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e498b069-8f67-4255-9008-774408fd1b8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:54.170224 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e498b069-8f67-4255-9008-774408fd1b8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:54.172759 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e498b069-8f67-4255-9008-774408fd1b8c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:54.209299 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1112/2224] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:21:54 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:21:54.209919 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b8ad4c-38ad-4c8b-a011-21ed3d134d6f admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:54.224948 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b8ad4c-38ad-4c8b-a011-21ed3d134d6f admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:54.236805 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8bfc1e-e6d4-47d9-bb88-58bdce300c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:54.238312 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8bfc1e-e6d4-47d9-bb88-58bdce300c69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:54.238647 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8bfc1e-e6d4-47d9-bb88-58bdce300c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:54.239006 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8bfc1e-e6d4-47d9-bb88-58bdce300c69 admin admin] PATH_INFO: `/v3/projects/2c3810402f544d4c96b81850cad47fd9/users/34774e9da9f948b48c123457bca2281d/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:54.273949 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8bfc1e-e6d4-47d9-bb88-58bdce300c69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c3810402f544d4c96b81850cad47fd9, user_id=34774e9da9f948b48c123457bca2281d, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:54.277975 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8bfc1e-e6d4-47d9-bb88-58bdce300c69 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:54.314099 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1113/2225] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:21:54 2022] PUT /identity/v3/projects/2c3810402f544d4c96b81850cad47fd9/users/34774e9da9f948b48c123457bca2281d/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:21:54.325557 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f23bc-65c9-4e72-9289-063e3cab63fd None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:54.325557 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f23bc-65c9-4e72-9289-063e3cab63fd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:54.325828 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f23bc-65c9-4e72-9289-063e3cab63fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:54.383731 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-646f23bc-65c9-4e72-9289-063e3cab63fd None None] MFA Rules not processed for user `34774e9da9f948b48c123457bca2281d`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:21:54.385407 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-646f23bc-65c9-4e72-9289-063e3cab63fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:54.509706 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1113/2226] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:21:54 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:21:54.519020 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8bfc1e-e6d4-47d9-bb88-58bdce300c69 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:54.520859 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-8b8bfc1e-e6d4-47d9-bb88-58bdce300c69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:54.564485 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d56c2ec-992d-4bf8-ad0d-adab83681161 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:54.564785 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d56c2ec-992d-4bf8-ad0d-adab83681161 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:54.565089 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d56c2ec-992d-4bf8-ad0d-adab83681161 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:54.566768 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1114/2227] 10.222.0.71 () {54 vars in 921 bytes} [Mon Jul 4 09:21:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:21:54.572968 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2dd3b7-254f-458c-b62d-77573056dede None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:54.573290 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2dd3b7-254f-458c-b62d-77573056dede None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:54.573679 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2dd3b7-254f-458c-b62d-77573056dede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:54.599539 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8bfc1e-e6d4-47d9-bb88-58bdce300c69 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:54.606081 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b009e91-7dfa-4814-b167-d54baac182e0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '192724527d774817bc4581474ac848dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:54.606774 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b009e91-7dfa-4814-b167-d54baac182e0 service glance] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:54.607512 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b009e91-7dfa-4814-b167-d54baac182e0 service glance] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:54.607982 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b009e91-7dfa-4814-b167-d54baac182e0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:54.615111 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7b2dd3b7-254f-458c-b62d-77573056dede None None] MFA Rules not processed for user `15039da209ec445d93d49e2bd6d854eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:21:54.617942 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7b2dd3b7-254f-458c-b62d-77573056dede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:54.622133 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b009e91-7dfa-4814-b167-d54baac182e0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:54.624218 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b009e91-7dfa-4814-b167-d54baac182e0 service glance] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:54.641171 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1114/2228] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:21:54 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:21:54.736715 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:21:54.765571 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1115/2229] 10.222.0.71 () {58 vars in 1018 bytes} [Mon Jul 4 09:21:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:21:54.786848 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b009e91-7dfa-4814-b167-d54baac182e0 service glance] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:54.792060 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-1b009e91-7dfa-4814-b167-d54baac182e0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:54.825981 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b009e91-7dfa-4814-b167-d54baac182e0 service glance] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:54.850039 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e111cf12-1af1-4a3d-88d7-ad9866af38a6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '192724527d774817bc4581474ac848dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:54.850872 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111cf12-1af1-4a3d-88d7-ad9866af38a6 service glance] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:54.851351 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111cf12-1af1-4a3d-88d7-ad9866af38a6 service glance] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:54.851725 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111cf12-1af1-4a3d-88d7-ad9866af38a6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:54.908169 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e111cf12-1af1-4a3d-88d7-ad9866af38a6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:54.913926 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e111cf12-1af1-4a3d-88d7-ad9866af38a6 service glance] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:54.944774 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1115/2230] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:21:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:21:55.053231 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9099de-6141-4b45-a93d-858c96eb7b06 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:55.053526 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9099de-6141-4b45-a93d-858c96eb7b06 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:55.053853 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9099de-6141-4b45-a93d-858c96eb7b06 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:55.055598 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1116/2231] 10.222.0.71 () {54 vars in 921 bytes} [Mon Jul 4 09:21:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:21:55.061992 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78248d44-dd03-4744-82fa-bad63bf80b82 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:55.062476 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78248d44-dd03-4744-82fa-bad63bf80b82 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:55.062924 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78248d44-dd03-4744-82fa-bad63bf80b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:55.096706 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-78248d44-dd03-4744-82fa-bad63bf80b82 None None] MFA Rules not processed for user `15818e58661646f381da0973d4c2ff64`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:21:55.097979 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-78248d44-dd03-4744-82fa-bad63bf80b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:55.160708 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:21:55.174445 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1116/2232] 10.222.0.71 () {58 vars in 1018 bytes} [Mon Jul 4 09:21:55 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:21:55.187412 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9099de-6141-4b45-a93d-858c96eb7b06 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:55.189936 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ee9099de-6141-4b45-a93d-858c96eb7b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:55.205204 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9099de-6141-4b45-a93d-858c96eb7b06 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:55.208756 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c8c99b-be9f-41d3-b35b-4532f2330671 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:55.209484 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c8c99b-be9f-41d3-b35b-4532f2330671 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:55.210292 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c8c99b-be9f-41d3-b35b-4532f2330671 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:55.210292 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c8c99b-be9f-41d3-b35b-4532f2330671 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:55.219786 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c8c99b-be9f-41d3-b35b-4532f2330671 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:55.221279 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c8c99b-be9f-41d3-b35b-4532f2330671 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:55.230299 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1117/2233] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:21:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:21:55.398766 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0c40a-f48f-45db-b9b2-7f535ae50d3d None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:55.399287 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0c40a-f48f-45db-b9b2-7f535ae50d3d None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:55.399625 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0c40a-f48f-45db-b9b2-7f535ae50d3d None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:55.402907 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1117/2234] 10.222.0.71 () {54 vars in 921 bytes} [Mon Jul 4 09:21:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:21:55.409498 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead3b30-9578-4cef-b7ce-4c3d9c46c888 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:55.409726 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead3b30-9578-4cef-b7ce-4c3d9c46c888 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:55.410008 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead3b30-9578-4cef-b7ce-4c3d9c46c888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:55.435537 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-dead3b30-9578-4cef-b7ce-4c3d9c46c888 None None] MFA Rules not processed for user `15039da209ec445d93d49e2bd6d854eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:21:55.438782 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dead3b30-9578-4cef-b7ce-4c3d9c46c888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:55.451472 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1118/2235] 10.222.0.71 () {58 vars in 1018 bytes} [Mon Jul 4 09:21:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:21:55.464037 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada0c40a-f48f-45db-b9b2-7f535ae50d3d None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:55.475652 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada0c40a-f48f-45db-b9b2-7f535ae50d3d None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:55.481726 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949fbbcc-b64c-4f6a-927f-d07f8f3288e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:55.482583 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949fbbcc-b64c-4f6a-927f-d07f8f3288e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:55.483078 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949fbbcc-b64c-4f6a-927f-d07f8f3288e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:55.483366 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949fbbcc-b64c-4f6a-927f-d07f8f3288e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:55.495924 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949fbbcc-b64c-4f6a-927f-d07f8f3288e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:55.497666 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949fbbcc-b64c-4f6a-927f-d07f8f3288e5 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:55.510424 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1118/2236] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:21:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:21:59.451829 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f6b8d-a61e-411a-8401-03f0f71b2179 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:59.452455 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f6b8d-a61e-411a-8401-03f0f71b2179 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:59.452455 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f6b8d-a61e-411a-8401-03f0f71b2179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:59.471170 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8f3f6b8d-a61e-411a-8401-03f0f71b2179 None None] MFA Rules not processed for user `78668b939c5847b1b73e9c28b5d8db7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:21:59.472905 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8f3f6b8d-a61e-411a-8401-03f0f71b2179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:59.562342 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1119/2237] 10.222.0.71 () {58 vars in 1009 bytes} [Mon Jul 4 09:21:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:21:59.576536 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949fbbcc-b64c-4f6a-927f-d07f8f3288e5 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:59.577117 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-949fbbcc-b64c-4f6a-927f-d07f8f3288e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:59.659569 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949fbbcc-b64c-4f6a-927f-d07f8f3288e5 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:59.664633 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c524ce-d1c5-451e-878a-3f22804882f7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88b41b55769a4c449716c3590ed58225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:59.665357 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c524ce-d1c5-451e-878a-3f22804882f7 service swift] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:59.665642 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c524ce-d1c5-451e-878a-3f22804882f7 service swift] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:59.665926 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c524ce-d1c5-451e-878a-3f22804882f7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:59.675482 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c524ce-d1c5-451e-878a-3f22804882f7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:59.677102 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c524ce-d1c5-451e-878a-3f22804882f7 service swift] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:59.687515 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1119/2238] 10.222.0.71 () {60 vars in 1384 bytes} [Mon Jul 4 09:21:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 09:22:09.555126 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=45 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:22:09.683788 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=41 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:22:12.495707 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3f6b8d-a61e-411a-8401-03f0f71b2179 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:22:12.497133 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8f3f6b8d-a61e-411a-8401-03f0f71b2179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:22:12.588313 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3f6b8d-a61e-411a-8401-03f0f71b2179 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:22:12.594269 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e618ad-39e3-4b47-9e8c-00bf5ca9583f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '293c695f15c8437d81a619f840318f44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:22:12.594950 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e618ad-39e3-4b47-9e8c-00bf5ca9583f service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:22:12.595237 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e618ad-39e3-4b47-9e8c-00bf5ca9583f service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:22:12.595516 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e618ad-39e3-4b47-9e8c-00bf5ca9583f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:22:12.597301 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c6e618ad-39e3-4b47-9e8c-00bf5ca9583f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:22:12.609234 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e618ad-39e3-4b47-9e8c-00bf5ca9583f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:22:12.610802 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e618ad-39e3-4b47-9e8c-00bf5ca9583f service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:22:12.620596 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1120/2239] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:22:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:22:22.616253 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=41 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:22:33.573407 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c524ce-d1c5-451e-878a-3f22804882f7 service swift] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:22:33.679258 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c524ce-d1c5-451e-878a-3f22804882f7 service swift] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:22:33.685856 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775d85f5-89b5-4cd2-b38a-93620939518d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:22:33.686834 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775d85f5-89b5-4cd2-b38a-93620939518d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:22:33.687058 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775d85f5-89b5-4cd2-b38a-93620939518d admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:22:33.687347 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775d85f5-89b5-4cd2-b38a-93620939518d admin admin] PATH_INFO: `/v3/users/34774e9da9f948b48c123457bca2281d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:22:33.688585 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775d85f5-89b5-4cd2-b38a-93620939518d admin admin] RBAC: Authorizing `identity:delete_user(user_id=34774e9da9f948b48c123457bca2281d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:22:33.690592 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775d85f5-89b5-4cd2-b38a-93620939518d admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:22:33.750999 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-775d85f5-89b5-4cd2-b38a-93620939518d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34774e9da9f948b48c123457bca2281d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:22:33.759176 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-775d85f5-89b5-4cd2-b38a-93620939518d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34774e9da9f948b48c123457bca2281d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:22:33.759875 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-775d85f5-89b5-4cd2-b38a-93620939518d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34774e9da9f948b48c123457bca2281d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:22:33.767856 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-775d85f5-89b5-4cd2-b38a-93620939518d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34774e9da9f948b48c123457bca2281d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:22:33.780135 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-775d85f5-89b5-4cd2-b38a-93620939518d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34774e9da9f948b48c123457bca2281d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:22:33.786428 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1120/2240] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:22:33 2022] DELETE /identity/v3/users/34774e9da9f948b48c123457bca2281d => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:22:33.790964 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e8127b-3ae6-48c4-a862-0305353013bb None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:22:33.791511 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e8127b-3ae6-48c4-a862-0305353013bb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:22:33.791606 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e8127b-3ae6-48c4-a862-0305353013bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:22:33.811344 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a1e8127b-3ae6-48c4-a862-0305353013bb None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:22:33.812684 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a1e8127b-3ae6-48c4-a862-0305353013bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:22:33.885145 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1121/2241] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:22:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:22:33.895861 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775d85f5-89b5-4cd2-b38a-93620939518d admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:22:33.897317 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-775d85f5-89b5-4cd2-b38a-93620939518d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:22:33.954211 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775d85f5-89b5-4cd2-b38a-93620939518d admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:22:33.957862 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584b0c0f-b483-4e3d-83d0-46445eee47c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:22:33.958504 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584b0c0f-b483-4e3d-83d0-46445eee47c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:22:33.958793 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584b0c0f-b483-4e3d-83d0-46445eee47c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:22:33.959071 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584b0c0f-b483-4e3d-83d0-46445eee47c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:22:33.968864 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584b0c0f-b483-4e3d-83d0-46445eee47c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:22:33.970351 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584b0c0f-b483-4e3d-83d0-46445eee47c7 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:22:33.979826 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1121/2242] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:22:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:22:34.265673 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e8127b-3ae6-48c4-a862-0305353013bb None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:22:34.267169 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a1e8127b-3ae6-48c4-a862-0305353013bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:22:34.281751 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e8127b-3ae6-48c4-a862-0305353013bb None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:22:34.287283 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c0a8ab-1ec7-4f9f-81bb-7d1d8d07c5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:22:34.288207 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c0a8ab-1ec7-4f9f-81bb-7d1d8d07c5e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:22:34.288519 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c0a8ab-1ec7-4f9f-81bb-7d1d8d07c5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:22:34.288877 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c0a8ab-1ec7-4f9f-81bb-7d1d8d07c5e9 admin admin] PATH_INFO: `/v3/projects/2c3810402f544d4c96b81850cad47fd9` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:22:34.290582 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c0a8ab-1ec7-4f9f-81bb-7d1d8d07c5e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c3810402f544d4c96b81850cad47fd9)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:22:34.293432 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c0a8ab-1ec7-4f9f-81bb-7d1d8d07c5e9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:22:34.342978 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e5c0a8ab-1ec7-4f9f-81bb-7d1d8d07c5e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c3810402f544d4c96b81850cad47fd9', 'request_id': 'req-e5c0a8ab-1ec7-4f9f-81bb-7d1d8d07c5e9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:22:34.348323 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e5c0a8ab-1ec7-4f9f-81bb-7d1d8d07c5e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c3810402f544d4c96b81850cad47fd9', 'request_id': 'req-e5c0a8ab-1ec7-4f9f-81bb-7d1d8d07c5e9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:22:34.354449 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e5c0a8ab-1ec7-4f9f-81bb-7d1d8d07c5e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c3810402f544d4c96b81850cad47fd9', 'request_id': 'req-e5c0a8ab-1ec7-4f9f-81bb-7d1d8d07c5e9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:22:34.366625 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e5c0a8ab-1ec7-4f9f-81bb-7d1d8d07c5e9 admin admin] The token cache is being invalidate because project 2c3810402f544d4c96b81850cad47fd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:22:34.366968 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e5c0a8ab-1ec7-4f9f-81bb-7d1d8d07c5e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:22:34.367536 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e5c0a8ab-1ec7-4f9f-81bb-7d1d8d07c5e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:22:34.369126 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1122/2243] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:22:34 2022] DELETE /identity/v3/projects/2c3810402f544d4c96b81850cad47fd9 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:22:37.352864 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19e591-a9ef-4e24-867c-a24b2d8a5713 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:22:37.353337 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19e591-a9ef-4e24-867c-a24b2d8a5713 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:22:37.353467 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19e591-a9ef-4e24-867c-a24b2d8a5713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:22:37.393345 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:22:37.421527 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-0d19e591-a9ef-4e24-867c-a24b2d8a5713 None None] MFA Rules not processed for user `78668b939c5847b1b73e9c28b5d8db7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:22:37.424066 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0d19e591-a9ef-4e24-867c-a24b2d8a5713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:22:37.504793 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1122/2244] 10.222.0.71 () {58 vars in 967 bytes} [Mon Jul 4 09:22:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:22:37.527879 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c0a8ab-1ec7-4f9f-81bb-7d1d8d07c5e9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:22:37.529850 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e5c0a8ab-1ec7-4f9f-81bb-7d1d8d07c5e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:22:37.631556 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c0a8ab-1ec7-4f9f-81bb-7d1d8d07c5e9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:22:37.636797 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26fbe81-6bdd-480e-a56f-395eacb9dd4a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88b41b55769a4c449716c3590ed58225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:22:37.637479 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26fbe81-6bdd-480e-a56f-395eacb9dd4a service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:22:37.637766 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26fbe81-6bdd-480e-a56f-395eacb9dd4a service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:22:37.638049 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26fbe81-6bdd-480e-a56f-395eacb9dd4a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:22:37.643678 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:22:37.648439 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26fbe81-6bdd-480e-a56f-395eacb9dd4a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:22:37.651058 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26fbe81-6bdd-480e-a56f-395eacb9dd4a service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:22:37.661409 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1123/2245] 10.222.0.71 () {60 vars in 1384 bytes} [Mon Jul 4 09:22:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 09:22:43.944929 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705d217-9c57-4abe-bbb3-2c0efe5b9366 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:22:43.945505 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705d217-9c57-4abe-bbb3-2c0efe5b9366 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:22:43.945505 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705d217-9c57-4abe-bbb3-2c0efe5b9366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:22:43.963513 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-1705d217-9c57-4abe-bbb3-2c0efe5b9366 None None] MFA Rules not processed for user `9bcf554f3d6447399e1d9cfe08b0bff7`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:22:43.964719 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-1705d217-9c57-4abe-bbb3-2c0efe5b9366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:22:44.020406 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1123/2246] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:22:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:22:47.658507 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:22:53.030127 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92596f03-4420-4b40-aeae-0baf5c385898 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:22:53.030743 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92596f03-4420-4b40-aeae-0baf5c385898 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:22:53.030743 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92596f03-4420-4b40-aeae-0baf5c385898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:22:53.047470 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-92596f03-4420-4b40-aeae-0baf5c385898 None None] MFA Rules not processed for user `6bc30b7d536444008e5b5bab45c5c438`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:22:53.048678 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-92596f03-4420-4b40-aeae-0baf5c385898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:22:53.103432 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1124/2247] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:22:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:22:54.013627 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=55 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:23:03.095996 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=25 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:23:13.579590 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705d217-9c57-4abe-bbb3-2c0efe5b9366 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:23:13.581139 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-1705d217-9c57-4abe-bbb3-2c0efe5b9366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:23:13.647628 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705d217-9c57-4abe-bbb3-2c0efe5b9366 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:23:13.652022 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1ba0a6-f21f-4e18-84b0-2536ffa606da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:23:13.652901 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ba0a6-f21f-4e18-84b0-2536ffa606da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:23:13.653217 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ba0a6-f21f-4e18-84b0-2536ffa606da admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:23:13.654153 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ba0a6-f21f-4e18-84b0-2536ffa606da admin admin] PATH_INFO: `/v3/users/65f1d0ab53de4473acee63b55bf14cab` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:23:13.655485 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1ba0a6-f21f-4e18-84b0-2536ffa606da admin admin] RBAC: Authorizing `identity:delete_user(user_id=65f1d0ab53de4473acee63b55bf14cab)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:23:13.657453 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1ba0a6-f21f-4e18-84b0-2536ffa606da admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:23:13.704887 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-df1ba0a6-f21f-4e18-84b0-2536ffa606da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65f1d0ab53de4473acee63b55bf14cab'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:23:13.712282 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-df1ba0a6-f21f-4e18-84b0-2536ffa606da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65f1d0ab53de4473acee63b55bf14cab'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:23:13.713243 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-df1ba0a6-f21f-4e18-84b0-2536ffa606da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65f1d0ab53de4473acee63b55bf14cab'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:23:13.720655 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-df1ba0a6-f21f-4e18-84b0-2536ffa606da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65f1d0ab53de4473acee63b55bf14cab'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:23:13.732589 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-df1ba0a6-f21f-4e18-84b0-2536ffa606da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65f1d0ab53de4473acee63b55bf14cab'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:23:13.739998 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1124/2248] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:23:13 2022] DELETE /identity/v3/users/65f1d0ab53de4473acee63b55bf14cab => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:23:13.928775 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92596f03-4420-4b40-aeae-0baf5c385898 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:23:13.941053 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92596f03-4420-4b40-aeae-0baf5c385898 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:23:13.998923 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc1992c-a47b-4566-a17e-d28d280aab87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:23:13.999775 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc1992c-a47b-4566-a17e-d28d280aab87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:23:14.000055 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc1992c-a47b-4566-a17e-d28d280aab87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:23:14.000326 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc1992c-a47b-4566-a17e-d28d280aab87 admin admin] PATH_INFO: `/v3/projects/96bc6b8b0235439587a3a5c9e837d708` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:23:14.001514 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc1992c-a47b-4566-a17e-d28d280aab87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96bc6b8b0235439587a3a5c9e837d708)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:23:14.003416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc1992c-a47b-4566-a17e-d28d280aab87 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:23:14.049504 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3dc1992c-a47b-4566-a17e-d28d280aab87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96bc6b8b0235439587a3a5c9e837d708', 'request_id': 'req-3dc1992c-a47b-4566-a17e-d28d280aab87'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:23:14.054773 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3dc1992c-a47b-4566-a17e-d28d280aab87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96bc6b8b0235439587a3a5c9e837d708', 'request_id': 'req-3dc1992c-a47b-4566-a17e-d28d280aab87'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:23:14.061637 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3dc1992c-a47b-4566-a17e-d28d280aab87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96bc6b8b0235439587a3a5c9e837d708', 'request_id': 'req-3dc1992c-a47b-4566-a17e-d28d280aab87'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:23:14.074430 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3dc1992c-a47b-4566-a17e-d28d280aab87 admin admin] The token cache is being invalidate because project 96bc6b8b0235439587a3a5c9e837d708 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:23:14.074718 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3dc1992c-a47b-4566-a17e-d28d280aab87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:23:14.075380 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3dc1992c-a47b-4566-a17e-d28d280aab87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:23:14.077443 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1125/2249] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:23:13 2022] DELETE /identity/v3/projects/96bc6b8b0235439587a3a5c9e837d708 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:23:14.082325 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10822514-34fe-440a-bc84-df569fb302d1 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:23:14.082614 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10822514-34fe-440a-bc84-df569fb302d1 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:23:14.083178 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10822514-34fe-440a-bc84-df569fb302d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:23:14.101924 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-10822514-34fe-440a-bc84-df569fb302d1 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:23:14.103140 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-10822514-34fe-440a-bc84-df569fb302d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:23:14.167084 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1125/2250] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:23:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:23:14.175897 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ddfd1a-8b1c-4c33-83c5-25f33be34cd1 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:23:14.176189 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ddfd1a-8b1c-4c33-83c5-25f33be34cd1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:23:14.176476 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ddfd1a-8b1c-4c33-83c5-25f33be34cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:23:14.196338 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a5ddfd1a-8b1c-4c33-83c5-25f33be34cd1 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:23:14.197563 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a5ddfd1a-8b1c-4c33-83c5-25f33be34cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:23:14.205842 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1126/2251] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:23:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:23:14.209986 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10822514-34fe-440a-bc84-df569fb302d1 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:23:14.220112 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10822514-34fe-440a-bc84-df569fb302d1 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:23:14.224892 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e209c8-1196-49f3-a074-d63e62d79014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:23:14.225572 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e209c8-1196-49f3-a074-d63e62d79014 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:23:14.225858 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e209c8-1196-49f3-a074-d63e62d79014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:23:14.226137 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e209c8-1196-49f3-a074-d63e62d79014 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:23:14.226872 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e209c8-1196-49f3-a074-d63e62d79014 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:23:14.227182 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e209c8-1196-49f3-a074-d63e62d79014 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:23:14.228788 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e209c8-1196-49f3-a074-d63e62d79014 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:23:14.240848 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1126/2252] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:23:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:23:14.244881 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ddfd1a-8b1c-4c33-83c5-25f33be34cd1 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:23:14.254850 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ddfd1a-8b1c-4c33-83c5-25f33be34cd1 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:23:14.259533 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464c7d02-e562-4f9b-9222-615c09d0a8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:23:14.260177 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c7d02-e562-4f9b-9222-615c09d0a8bc admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:23:14.260462 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c7d02-e562-4f9b-9222-615c09d0a8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:23:14.260746 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c7d02-e562-4f9b-9222-615c09d0a8bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:23:14.261461 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464c7d02-e562-4f9b-9222-615c09d0a8bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:23:14.263574 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464c7d02-e562-4f9b-9222-615c09d0a8bc admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:23:14.287855 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1127/2253] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:23:14 2022] POST /identity/v3/projects => generated 397 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:23:14.291827 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e209c8-1196-49f3-a074-d63e62d79014 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:23:14.302622 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e209c8-1196-49f3-a074-d63e62d79014 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:23:14.306271 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:23:14.362728 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe26a31a-a8ed-404b-ba6a-50b0d8b11d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:23:14.363383 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26a31a-a8ed-404b-ba6a-50b0d8b11d95 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:23:14.363674 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26a31a-a8ed-404b-ba6a-50b0d8b11d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:23:14.363961 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26a31a-a8ed-404b-ba6a-50b0d8b11d95 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:23:14.364661 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe26a31a-a8ed-404b-ba6a-50b0d8b11d95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:23:14.366625 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe26a31a-a8ed-404b-ba6a-50b0d8b11d95 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:23:14.390512 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1127/2254] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:23:14 2022] POST /identity/v3/users => generated 299 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:23:14.394654 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464c7d02-e562-4f9b-9222-615c09d0a8bc admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:23:14.405394 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464c7d02-e562-4f9b-9222-615c09d0a8bc admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:23:14.410019 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d80d615-8872-48cc-a1e0-97bb24b16ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:23:14.410678 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80d615-8872-48cc-a1e0-97bb24b16ce6 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:23:14.410965 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80d615-8872-48cc-a1e0-97bb24b16ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:23:14.411246 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80d615-8872-48cc-a1e0-97bb24b16ce6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:23:14.411975 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d80d615-8872-48cc-a1e0-97bb24b16ce6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:23:14.413599 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d80d615-8872-48cc-a1e0-97bb24b16ce6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:23:14.424744 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1128/2255] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:23:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:23:14.428656 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe26a31a-a8ed-404b-ba6a-50b0d8b11d95 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:23:14.439764 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe26a31a-a8ed-404b-ba6a-50b0d8b11d95 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:23:14.444481 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2858cee6-a0ee-43f2-a8b9-43660ded2481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:23:14.445180 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2858cee6-a0ee-43f2-a8b9-43660ded2481 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:23:14.445470 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2858cee6-a0ee-43f2-a8b9-43660ded2481 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:23:14.445756 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2858cee6-a0ee-43f2-a8b9-43660ded2481 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:23:14.446492 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2858cee6-a0ee-43f2-a8b9-43660ded2481 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:23:14.448088 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2858cee6-a0ee-43f2-a8b9-43660ded2481 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:23:14.458416 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1128/2256] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:23:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:23:14.462425 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d80d615-8872-48cc-a1e0-97bb24b16ce6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:23:14.473430 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d80d615-8872-48cc-a1e0-97bb24b16ce6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:23:14.477987 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0743e004-3791-4f8b-a3d5-469e9d5f9a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:23:14.478851 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743e004-3791-4f8b-a3d5-469e9d5f9a6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:23:14.479137 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743e004-3791-4f8b-a3d5-469e9d5f9a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:23:14.479414 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743e004-3791-4f8b-a3d5-469e9d5f9a6b admin admin] PATH_INFO: `/v3/projects/d18521477bda45bfa822b849dac26fbc/users/7ae24949b1fc48e3a732a533ee6fad18/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:23:14.494571 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0743e004-3791-4f8b-a3d5-469e9d5f9a6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d18521477bda45bfa822b849dac26fbc, user_id=7ae24949b1fc48e3a732a533ee6fad18, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:23:14.497260 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0743e004-3791-4f8b-a3d5-469e9d5f9a6b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:23:14.510482 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1129/2257] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:23:14 2022] PUT /identity/v3/projects/d18521477bda45bfa822b849dac26fbc/users/7ae24949b1fc48e3a732a533ee6fad18/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:23:14.574937 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a9643-6f0b-442a-a613-562434b1ebff None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:23:14.575230 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a9643-6f0b-442a-a613-562434b1ebff None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:23:14.575754 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a9643-6f0b-442a-a613-562434b1ebff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:23:14.603191 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-465a9643-6f0b-442a-a613-562434b1ebff None None] MFA Rules not processed for user `7ae24949b1fc48e3a732a533ee6fad18`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:23:14.604406 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-465a9643-6f0b-442a-a613-562434b1ebff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:23:14.667413 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1129/2258] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:23:14 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:23:14.679615 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0743e004-3791-4f8b-a3d5-469e9d5f9a6b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:23:14.681288 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0743e004-3791-4f8b-a3d5-469e9d5f9a6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:23:14.711499 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:23:14.762146 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0743e004-3791-4f8b-a3d5-469e9d5f9a6b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:23:14.765995 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e48348-f0d2-4f00-85ba-64fed737cdac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:23:14.766600 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e48348-f0d2-4f00-85ba-64fed737cdac service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:23:14.766889 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e48348-f0d2-4f00-85ba-64fed737cdac service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:23:14.767170 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e48348-f0d2-4f00-85ba-64fed737cdac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:23:14.775696 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e48348-f0d2-4f00-85ba-64fed737cdac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:23:14.777134 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e48348-f0d2-4f00-85ba-64fed737cdac service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:23:14.786954 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1130/2259] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:23:14 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:23:22.828334 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465a9643-6f0b-442a-a613-562434b1ebff None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:23:22.892870 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465a9643-6f0b-442a-a613-562434b1ebff None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:23:22.897484 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1109030-720e-41dc-aa91-7762ab08e9f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:23:22.898355 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1109030-720e-41dc-aa91-7762ab08e9f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:23:22.898667 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1109030-720e-41dc-aa91-7762ab08e9f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:23:22.898946 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1109030-720e-41dc-aa91-7762ab08e9f0 admin admin] PATH_INFO: `/v3/users/7ae24949b1fc48e3a732a533ee6fad18` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:23:22.900153 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1109030-720e-41dc-aa91-7762ab08e9f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ae24949b1fc48e3a732a533ee6fad18)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:23:22.902126 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1109030-720e-41dc-aa91-7762ab08e9f0 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:23:22.917750 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:23:22.948012 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f1109030-720e-41dc-aa91-7762ab08e9f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ae24949b1fc48e3a732a533ee6fad18'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:23:22.956639 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f1109030-720e-41dc-aa91-7762ab08e9f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ae24949b1fc48e3a732a533ee6fad18'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:23:22.957315 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f1109030-720e-41dc-aa91-7762ab08e9f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ae24949b1fc48e3a732a533ee6fad18'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:23:22.964349 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f1109030-720e-41dc-aa91-7762ab08e9f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ae24949b1fc48e3a732a533ee6fad18'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:23:22.976940 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f1109030-720e-41dc-aa91-7762ab08e9f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ae24949b1fc48e3a732a533ee6fad18'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:23:22.982961 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1130/2260] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:23:22 2022] DELETE /identity/v3/users/7ae24949b1fc48e3a732a533ee6fad18 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:23:22.987483 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66323884-0067-4982-a5f0-8d4789a5fec4 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:23:22.987964 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66323884-0067-4982-a5f0-8d4789a5fec4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:23:22.988071 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66323884-0067-4982-a5f0-8d4789a5fec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:23:23.005901 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-66323884-0067-4982-a5f0-8d4789a5fec4 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:23:23.007108 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-66323884-0067-4982-a5f0-8d4789a5fec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:23:23.067288 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1131/2261] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:23:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:23:23.077874 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1109030-720e-41dc-aa91-7762ab08e9f0 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:23:23.079264 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f1109030-720e-41dc-aa91-7762ab08e9f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:23:23.136334 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1109030-720e-41dc-aa91-7762ab08e9f0 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:23:23.140545 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e2710d-e16b-4525-9dec-961569169f22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:23:23.141221 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e2710d-e16b-4525-9dec-961569169f22 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:23:23.141529 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e2710d-e16b-4525-9dec-961569169f22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:23:23.141789 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e2710d-e16b-4525-9dec-961569169f22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:23:23.150723 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e2710d-e16b-4525-9dec-961569169f22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:23:23.152155 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e2710d-e16b-4525-9dec-961569169f22 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:23:23.162183 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1131/2262] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:23:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:23:23.432752 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66323884-0067-4982-a5f0-8d4789a5fec4 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:23:23.434236 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-66323884-0067-4982-a5f0-8d4789a5fec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:23:23.447527 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66323884-0067-4982-a5f0-8d4789a5fec4 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:23:23.452977 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792bd50c-8858-4487-b975-0c0de71e806d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:23:23.453890 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792bd50c-8858-4487-b975-0c0de71e806d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:23:23.454178 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792bd50c-8858-4487-b975-0c0de71e806d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:23:23.454463 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792bd50c-8858-4487-b975-0c0de71e806d admin admin] PATH_INFO: `/v3/projects/d18521477bda45bfa822b849dac26fbc` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:23:23.455649 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792bd50c-8858-4487-b975-0c0de71e806d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d18521477bda45bfa822b849dac26fbc)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:23:23.457597 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792bd50c-8858-4487-b975-0c0de71e806d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:23:23.505160 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-792bd50c-8858-4487-b975-0c0de71e806d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd18521477bda45bfa822b849dac26fbc', 'request_id': 'req-792bd50c-8858-4487-b975-0c0de71e806d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:23:23.510541 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-792bd50c-8858-4487-b975-0c0de71e806d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd18521477bda45bfa822b849dac26fbc', 'request_id': 'req-792bd50c-8858-4487-b975-0c0de71e806d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:23:23.517490 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-792bd50c-8858-4487-b975-0c0de71e806d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd18521477bda45bfa822b849dac26fbc', 'request_id': 'req-792bd50c-8858-4487-b975-0c0de71e806d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:23:23.529373 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-792bd50c-8858-4487-b975-0c0de71e806d admin admin] The token cache is being invalidate because project d18521477bda45bfa822b849dac26fbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:23:23.529665 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-792bd50c-8858-4487-b975-0c0de71e806d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:23:23.530334 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-792bd50c-8858-4487-b975-0c0de71e806d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:23:23.531921 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1132/2263] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:23:23 2022] DELETE /identity/v3/projects/d18521477bda45bfa822b849dac26fbc => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:23:23.536935 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93f0da-83b5-481c-bec3-ae8ae16a035c None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:23:23.537267 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93f0da-83b5-481c-bec3-ae8ae16a035c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:23:23.537548 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93f0da-83b5-481c-bec3-ae8ae16a035c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:23:23.555224 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-4d93f0da-83b5-481c-bec3-ae8ae16a035c None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:23:23.556463 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4d93f0da-83b5-481c-bec3-ae8ae16a035c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:23:23.616170 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1132/2264] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:23:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:23:23.624423 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e9502-39b9-4062-8d00-0f371731bf80 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:23:23.624715 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e9502-39b9-4062-8d00-0f371731bf80 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:23:23.624995 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e9502-39b9-4062-8d00-0f371731bf80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:23:23.643928 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b38e9502-39b9-4062-8d00-0f371731bf80 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:23:23.645185 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b38e9502-39b9-4062-8d00-0f371731bf80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:23:23.653171 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1133/2265] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:23:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:23:23.657693 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d93f0da-83b5-481c-bec3-ae8ae16a035c None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:23:23.667538 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d93f0da-83b5-481c-bec3-ae8ae16a035c None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:23:23.672624 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe05b29-174b-41d8-8e5b-9bb279704def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:23:23.673332 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe05b29-174b-41d8-8e5b-9bb279704def admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:23:23.673623 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe05b29-174b-41d8-8e5b-9bb279704def admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:23:23.673904 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe05b29-174b-41d8-8e5b-9bb279704def admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:23:23.674635 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe05b29-174b-41d8-8e5b-9bb279704def admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:23:23.674945 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe05b29-174b-41d8-8e5b-9bb279704def admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:23:23.676553 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe05b29-174b-41d8-8e5b-9bb279704def admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:23:23.689206 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1133/2266] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:23:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:23:23.693513 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38e9502-39b9-4062-8d00-0f371731bf80 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:23:23.703129 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38e9502-39b9-4062-8d00-0f371731bf80 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:23:23.707763 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d8233b-5edc-41ae-9474-db265eab030f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:23:23.708423 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8233b-5edc-41ae-9474-db265eab030f admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:23:23.708706 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8233b-5edc-41ae-9474-db265eab030f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:23:23.708983 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8233b-5edc-41ae-9474-db265eab030f admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:23:23.709739 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d8233b-5edc-41ae-9474-db265eab030f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:23:23.711633 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d8233b-5edc-41ae-9474-db265eab030f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:23:23.741970 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1134/2267] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:23:23 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:23:23.743161 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe05b29-174b-41d8-8e5b-9bb279704def admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:23:23.753642 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe05b29-174b-41d8-8e5b-9bb279704def admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:23:23.774085 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:23:23.814344 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8674198a-75c4-4ba1-b4ec-9e46bdb56467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:23:23.815077 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674198a-75c4-4ba1-b4ec-9e46bdb56467 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:23:23.815419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674198a-75c4-4ba1-b4ec-9e46bdb56467 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:23:23.815768 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674198a-75c4-4ba1-b4ec-9e46bdb56467 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:23:23.816701 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8674198a-75c4-4ba1-b4ec-9e46bdb56467 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:23:23.819427 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8674198a-75c4-4ba1-b4ec-9e46bdb56467 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:23:23.839215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1134/2268] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:23:23 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:23:23.843893 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d8233b-5edc-41ae-9474-db265eab030f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:23:23.850410 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:23:23.854761 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d8233b-5edc-41ae-9474-db265eab030f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:23:23.861615 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0d288f-ea7f-4340-868d-bd03ff5b2f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:23:23.862401 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d288f-ea7f-4340-868d-bd03ff5b2f0c admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:23:23.862753 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d288f-ea7f-4340-868d-bd03ff5b2f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:23:23.863183 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d288f-ea7f-4340-868d-bd03ff5b2f0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:23:23.863915 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0d288f-ea7f-4340-868d-bd03ff5b2f0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:23:23.865563 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0d288f-ea7f-4340-868d-bd03ff5b2f0c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:23:23.876690 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1135/2269] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:23:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:23:23.880790 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8674198a-75c4-4ba1-b4ec-9e46bdb56467 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:23:23.891002 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8674198a-75c4-4ba1-b4ec-9e46bdb56467 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:23:23.895724 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daa2cc5-5d6c-45e9-ab97-58720460b3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:23:23.896387 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa2cc5-5d6c-45e9-ab97-58720460b3a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:23:23.896673 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa2cc5-5d6c-45e9-ab97-58720460b3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:23:23.896955 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa2cc5-5d6c-45e9-ab97-58720460b3a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:23:23.897727 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daa2cc5-5d6c-45e9-ab97-58720460b3a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:23:23.899358 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daa2cc5-5d6c-45e9-ab97-58720460b3a9 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:23:23.910110 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1135/2270] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:23:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:23:23.913981 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0d288f-ea7f-4340-868d-bd03ff5b2f0c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:23:23.924242 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0d288f-ea7f-4340-868d-bd03ff5b2f0c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:23:23.928973 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd9f6e6-3ce0-40ce-a49f-bfae863cc15c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:23:23.929900 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9f6e6-3ce0-40ce-a49f-bfae863cc15c admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:23:23.930187 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9f6e6-3ce0-40ce-a49f-bfae863cc15c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:23:23.930472 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9f6e6-3ce0-40ce-a49f-bfae863cc15c admin admin] PATH_INFO: `/v3/projects/c1411a14411442a1bded43af5f67decd/users/b5ed81725b574b01aba6c604fbf9ed50/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:23:23.946249 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd9f6e6-3ce0-40ce-a49f-bfae863cc15c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1411a14411442a1bded43af5f67decd, user_id=b5ed81725b574b01aba6c604fbf9ed50, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:23:23.948890 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd9f6e6-3ce0-40ce-a49f-bfae863cc15c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:23:23.961700 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1136/2271] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:23:23 2022] PUT /identity/v3/projects/c1411a14411442a1bded43af5f67decd/users/b5ed81725b574b01aba6c604fbf9ed50/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:23:23.966991 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95da9ed-f23b-4986-b204-2bab5fd1b287 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:23:23.967279 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95da9ed-f23b-4986-b204-2bab5fd1b287 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:23:23.967564 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95da9ed-f23b-4986-b204-2bab5fd1b287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:23:23.985119 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-f95da9ed-f23b-4986-b204-2bab5fd1b287 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:23:23.986277 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f95da9ed-f23b-4986-b204-2bab5fd1b287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:23:24.044127 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1136/2272] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:23:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:23:24.054633 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd9f6e6-3ce0-40ce-a49f-bfae863cc15c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:23:24.056331 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1dd9f6e6-3ce0-40ce-a49f-bfae863cc15c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:23:24.114067 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd9f6e6-3ce0-40ce-a49f-bfae863cc15c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:23:24.117713 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797fbf25-81a0-437a-9af7-3c4a2d1128c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:23:24.118359 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797fbf25-81a0-437a-9af7-3c4a2d1128c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:23:24.118646 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797fbf25-81a0-437a-9af7-3c4a2d1128c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:23:24.118924 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797fbf25-81a0-437a-9af7-3c4a2d1128c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:23:24.128921 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797fbf25-81a0-437a-9af7-3c4a2d1128c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:23:24.130320 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797fbf25-81a0-437a-9af7-3c4a2d1128c1 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:23:24.139686 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1137/2273] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:23:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:23:28.591855 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf8e888-dd99-488f-989b-ca8e432577c3 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:23:28.592622 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf8e888-dd99-488f-989b-ca8e432577c3 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:23:28.592622 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf8e888-dd99-488f-989b-ca8e432577c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:23:28.633763 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-ebf8e888-dd99-488f-989b-ca8e432577c3 None None] MFA Rules not processed for user `b5ed81725b574b01aba6c604fbf9ed50`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:23:28.635081 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ebf8e888-dd99-488f-989b-ca8e432577c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:23:28.717372 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1137/2274] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:23:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:23:28.718598 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797fbf25-81a0-437a-9af7-3c4a2d1128c1 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:23:28.728986 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797fbf25-81a0-437a-9af7-3c4a2d1128c1 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:23:28.736374 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00082b8c-41b3-4e78-889c-b9a6aef28524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:23:28.737193 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00082b8c-41b3-4e78-889c-b9a6aef28524 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:23:28.737579 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00082b8c-41b3-4e78-889c-b9a6aef28524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:23:28.738086 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00082b8c-41b3-4e78-889c-b9a6aef28524 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:23:28.739267 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00082b8c-41b3-4e78-889c-b9a6aef28524 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:23:28.741843 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00082b8c-41b3-4e78-889c-b9a6aef28524 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:23:28.779816 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1138/2275] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:23:28 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:23:28.781691 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf8e888-dd99-488f-989b-ca8e432577c3 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:23:28.793364 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf8e888-dd99-488f-989b-ca8e432577c3 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:23:28.827364 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:23:28.882655 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23178e96-54bf-4933-9098-8558882105f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:23:28.883634 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23178e96-54bf-4933-9098-8558882105f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:23:28.884108 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23178e96-54bf-4933-9098-8558882105f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:23:28.884899 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23178e96-54bf-4933-9098-8558882105f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:23:28.885922 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23178e96-54bf-4933-9098-8558882105f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:23:28.887919 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23178e96-54bf-4933-9098-8558882105f9 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:23:28.924421 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1138/2276] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:23:28 2022] POST /identity/v3/users => generated 298 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:23:28.928372 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00082b8c-41b3-4e78-889c-b9a6aef28524 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:23:28.939845 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00082b8c-41b3-4e78-889c-b9a6aef28524 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:23:28.944690 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8366aa-0434-4221-98a6-9233a51002e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:23:28.945664 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8366aa-0434-4221-98a6-9233a51002e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:23:28.946137 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8366aa-0434-4221-98a6-9233a51002e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:23:28.946620 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8366aa-0434-4221-98a6-9233a51002e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:23:28.947701 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8366aa-0434-4221-98a6-9233a51002e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:23:28.949571 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8366aa-0434-4221-98a6-9233a51002e9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:23:28.964838 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1139/2277] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:23:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:23:28.966388 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23178e96-54bf-4933-9098-8558882105f9 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:23:28.976880 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23178e96-54bf-4933-9098-8558882105f9 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:23:28.983126 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4cb012-b297-4d1a-8187-620494e73147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:23:28.984208 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4cb012-b297-4d1a-8187-620494e73147 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:23:28.984482 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4cb012-b297-4d1a-8187-620494e73147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:23:28.984824 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4cb012-b297-4d1a-8187-620494e73147 admin admin] PATH_INFO: `/v3/projects/bf83ec218a6846478ec086eda44cca45/users/db53b4ed907444a19288bef6dec6a426/roles/f36e053b442540f08041ac7878760f65` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:23:29.003051 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4cb012-b297-4d1a-8187-620494e73147 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf83ec218a6846478ec086eda44cca45, user_id=db53b4ed907444a19288bef6dec6a426, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:23:29.005898 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4cb012-b297-4d1a-8187-620494e73147 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:23:29.022201 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1139/2278] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:23:28 2022] PUT /identity/v3/projects/bf83ec218a6846478ec086eda44cca45/users/db53b4ed907444a19288bef6dec6a426/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:23:29.026235 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8366aa-0434-4221-98a6-9233a51002e9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:23:29.039721 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8366aa-0434-4221-98a6-9233a51002e9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:23:29.129223 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc17201-d7c7-438d-90af-6942c332b803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:23:29.129922 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc17201-d7c7-438d-90af-6942c332b803 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:23:29.130565 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc17201-d7c7-438d-90af-6942c332b803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:23:29.130892 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc17201-d7c7-438d-90af-6942c332b803 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:23:29.131883 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc17201-d7c7-438d-90af-6942c332b803 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:23:29.133535 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc17201-d7c7-438d-90af-6942c332b803 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:23:29.147199 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1140/2279] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:23:29 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:23:29.151707 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4cb012-b297-4d1a-8187-620494e73147 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:23:29.163547 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4cb012-b297-4d1a-8187-620494e73147 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:23:29.168959 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fea5c2-cbf0-4fd2-88cf-0fef1d4e2dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:23:29.169939 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fea5c2-cbf0-4fd2-88cf-0fef1d4e2dcf admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:23:29.170227 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fea5c2-cbf0-4fd2-88cf-0fef1d4e2dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:23:29.170508 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fea5c2-cbf0-4fd2-88cf-0fef1d4e2dcf admin admin] PATH_INFO: `/v3/projects/bf83ec218a6846478ec086eda44cca45/users/db53b4ed907444a19288bef6dec6a426/roles/f36e053b442540f08041ac7878760f65` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:23:29.172718 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fea5c2-cbf0-4fd2-88cf-0fef1d4e2dcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf83ec218a6846478ec086eda44cca45, user_id=db53b4ed907444a19288bef6dec6a426, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:23:29.176311 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fea5c2-cbf0-4fd2-88cf-0fef1d4e2dcf admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:23:29.191817 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1140/2280] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:23:29 2022] PUT /identity/v3/projects/bf83ec218a6846478ec086eda44cca45/users/db53b4ed907444a19288bef6dec6a426/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:23:29.724170 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a896067f-9a08-4c66-993e-e06a4c6314ef None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:23:29.724715 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a896067f-9a08-4c66-993e-e06a4c6314ef None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:23:29.724715 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a896067f-9a08-4c66-993e-e06a4c6314ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:23:29.759266 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a896067f-9a08-4c66-993e-e06a4c6314ef None None] MFA Rules not processed for user `60d06d081a29460b8e85b0b301399f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:23:29.760694 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a896067f-9a08-4c66-993e-e06a4c6314ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:23:29.798529 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:23:29.831149 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1141/2281] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:23:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:23:34.770172 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2754e2-ea73-463b-a0b0-15351c9bf2fa None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:23:34.770644 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2754e2-ea73-463b-a0b0-15351c9bf2fa None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:23:34.770765 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2754e2-ea73-463b-a0b0-15351c9bf2fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:23:34.809644 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-4c2754e2-ea73-463b-a0b0-15351c9bf2fa None None] MFA Rules not processed for user `db53b4ed907444a19288bef6dec6a426`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:23:34.811797 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4c2754e2-ea73-463b-a0b0-15351c9bf2fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:23:34.894775 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1141/2282] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:23:34 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:23:34.908043 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a896067f-9a08-4c66-993e-e06a4c6314ef None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:23:34.909580 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a896067f-9a08-4c66-993e-e06a4c6314ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:23:35.021792 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a896067f-9a08-4c66-993e-e06a4c6314ef None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:23:35.025558 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf95a3e-2921-43c4-b823-0fd5fb6a17f6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '192724527d774817bc4581474ac848dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:23:35.026211 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf95a3e-2921-43c4-b823-0fd5fb6a17f6 service glance] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:23:35.026502 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf95a3e-2921-43c4-b823-0fd5fb6a17f6 service glance] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:23:35.026772 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf95a3e-2921-43c4-b823-0fd5fb6a17f6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:23:35.084389 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf95a3e-2921-43c4-b823-0fd5fb6a17f6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:23:35.085918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf95a3e-2921-43c4-b823-0fd5fb6a17f6 service glance] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:23:35.096042 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1142/2283] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:23:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:23:35.204924 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb22381-a0a8-4c57-99d5-3888afe0c7f8 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:23:35.205322 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb22381-a0a8-4c57-99d5-3888afe0c7f8 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:23:35.205536 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb22381-a0a8-4c57-99d5-3888afe0c7f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:23:35.228764 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-7bb22381-a0a8-4c57-99d5-3888afe0c7f8 None None] MFA Rules not processed for user `78668b939c5847b1b73e9c28b5d8db7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:23:35.230068 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7bb22381-a0a8-4c57-99d5-3888afe0c7f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:23:35.305795 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1142/2284] 10.222.0.71 () {58 vars in 1009 bytes} [Mon Jul 4 09:23:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:23:35.320524 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf95a3e-2921-43c4-b823-0fd5fb6a17f6 service glance] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:23:35.322113 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-eaf95a3e-2921-43c4-b823-0fd5fb6a17f6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:23:35.408172 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf95a3e-2921-43c4-b823-0fd5fb6a17f6 service glance] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:23:35.414960 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b28ed8-ad78-4253-b0f7-36fb5d53a9ce service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88b41b55769a4c449716c3590ed58225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:23:35.415660 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b28ed8-ad78-4253-b0f7-36fb5d53a9ce service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:23:35.415941 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b28ed8-ad78-4253-b0f7-36fb5d53a9ce service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:23:35.416216 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b28ed8-ad78-4253-b0f7-36fb5d53a9ce service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:23:35.426763 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b28ed8-ad78-4253-b0f7-36fb5d53a9ce service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:23:35.428432 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b28ed8-ad78-4253-b0f7-36fb5d53a9ce service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:23:35.439276 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1143/2285] 10.222.0.71 () {60 vars in 1384 bytes} [Mon Jul 4 09:23:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 09:23:44.868743 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d6e18-a89f-44ff-811f-6e8bf102058a None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:23:44.869319 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d6e18-a89f-44ff-811f-6e8bf102058a None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:23:44.869462 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d6e18-a89f-44ff-811f-6e8bf102058a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:23:44.889885 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-3b1d6e18-a89f-44ff-811f-6e8bf102058a None None] MFA Rules not processed for user `b5ed81725b574b01aba6c604fbf9ed50`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:23:44.891566 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-3b1d6e18-a89f-44ff-811f-6e8bf102058a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:23:44.902408 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1143/2286] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:23:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:23:45.435567 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:23:47.396602 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca735d8f-c20d-4418-b3e3-608980f16c31 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:23:47.397172 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca735d8f-c20d-4418-b3e3-608980f16c31 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:23:47.397172 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca735d8f-c20d-4418-b3e3-608980f16c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:23:47.415644 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ca735d8f-c20d-4418-b3e3-608980f16c31 None None] MFA Rules not processed for user `78668b939c5847b1b73e9c28b5d8db7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:23:47.416890 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ca735d8f-c20d-4418-b3e3-608980f16c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:23:47.424697 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1144/2287] 10.222.0.71 () {58 vars in 1009 bytes} [Mon Jul 4 09:23:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:23:47.437413 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1d6e18-a89f-44ff-811f-6e8bf102058a None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:23:47.457686 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1d6e18-a89f-44ff-811f-6e8bf102058a None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:23:47.465692 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f46fd46-9121-4a47-b3b7-3b7311413b69 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88b41b55769a4c449716c3590ed58225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:23:47.467387 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f46fd46-9121-4a47-b3b7-3b7311413b69 service swift] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:23:47.467831 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f46fd46-9121-4a47-b3b7-3b7311413b69 service swift] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:23:47.468263 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f46fd46-9121-4a47-b3b7-3b7311413b69 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:23:47.477725 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:23:47.481896 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f46fd46-9121-4a47-b3b7-3b7311413b69 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:23:47.485534 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f46fd46-9121-4a47-b3b7-3b7311413b69 service swift] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:23:47.500096 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1144/2288] 10.222.0.71 () {60 vars in 1384 bytes} [Mon Jul 4 09:23:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 09:23:57.410515 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:23:57.493489 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:24:02.661759 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca735d8f-c20d-4418-b3e3-608980f16c31 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:24:02.732723 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca735d8f-c20d-4418-b3e3-608980f16c31 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:24:02.739175 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4a4698-fe27-4445-9f6c-14f9c6b38db8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:24:02.739926 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a4698-fe27-4445-9f6c-14f9c6b38db8 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:24:02.740217 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a4698-fe27-4445-9f6c-14f9c6b38db8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:24:02.740553 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a4698-fe27-4445-9f6c-14f9c6b38db8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:24:02.742458 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0f4a4698-fe27-4445-9f6c-14f9c6b38db8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:24:02.850298 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4a4698-fe27-4445-9f6c-14f9c6b38db8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:24:02.851746 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4a4698-fe27-4445-9f6c-14f9c6b38db8 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:24:02.884860 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1145/2289] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:24:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:24:03.585188 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7391a934-4c2b-4b0d-bf26-fd13ea2186e6 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:24:03.585691 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7391a934-4c2b-4b0d-bf26-fd13ea2186e6 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:24:03.585797 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7391a934-4c2b-4b0d-bf26-fd13ea2186e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:24:03.603943 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-7391a934-4c2b-4b0d-bf26-fd13ea2186e6 None None] MFA Rules not processed for user `e50f9b53a10a43fa9b5fec03d01f451f`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:24:03.605189 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7391a934-4c2b-4b0d-bf26-fd13ea2186e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:24:03.659670 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1145/2290] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:24:03 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:24:03.695851 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4a4698-fe27-4445-9f6c-14f9c6b38db8 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:24:03.705691 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4a4698-fe27-4445-9f6c-14f9c6b38db8 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:24:03.709243 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96eba9c-9ba2-4a4c-9311-bb88ced194f6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '192724527d774817bc4581474ac848dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:24:03.709966 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96eba9c-9ba2-4a4c-9311-bb88ced194f6 service glance] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:24:03.710260 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96eba9c-9ba2-4a4c-9311-bb88ced194f6 service glance] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:24:03.710538 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96eba9c-9ba2-4a4c-9311-bb88ced194f6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:24:03.712299 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e96eba9c-9ba2-4a4c-9311-bb88ced194f6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:24:03.726073 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96eba9c-9ba2-4a4c-9311-bb88ced194f6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:24:03.727487 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96eba9c-9ba2-4a4c-9311-bb88ced194f6 service glance] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:24:03.738478 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1146/2291] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:24:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:24:13.652136 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=25 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:24:13.733705 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=69 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:24:14.304043 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7391a934-4c2b-4b0d-bf26-fd13ea2186e6 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:24:14.305773 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7391a934-4c2b-4b0d-bf26-fd13ea2186e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:24:14.322848 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7391a934-4c2b-4b0d-bf26-fd13ea2186e6 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:24:14.328445 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6190d4-3dfd-4235-950c-ce67d87abbb8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15039da209ec445d93d49e2bd6d854eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:24:14.329142 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6190d4-3dfd-4235-950c-ce67d87abbb8 service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:24:14.329660 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6190d4-3dfd-4235-950c-ce67d87abbb8 service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:24:14.329951 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6190d4-3dfd-4235-950c-ce67d87abbb8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:24:14.334113 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ac6190d4-3dfd-4235-950c-ce67d87abbb8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:24:14.400564 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6190d4-3dfd-4235-950c-ce67d87abbb8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:24:14.402198 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6190d4-3dfd-4235-950c-ce67d87abbb8 service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:24:14.412813 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1146/2292] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:24:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:24:19.344785 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96eba9c-9ba2-4a4c-9311-bb88ced194f6 service glance] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:24:19.355446 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96eba9c-9ba2-4a4c-9311-bb88ced194f6 service glance] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:24:19.359754 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7df4e61-641c-44e8-846c-3b22d4b76e40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:24:19.361076 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df4e61-641c-44e8-846c-3b22d4b76e40 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:24:19.361076 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df4e61-641c-44e8-846c-3b22d4b76e40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:24:19.361076 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df4e61-641c-44e8-846c-3b22d4b76e40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:24:19.372167 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7df4e61-641c-44e8-846c-3b22d4b76e40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:24:19.373726 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7df4e61-641c-44e8-846c-3b22d4b76e40 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:24:19.393899 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1147/2293] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:24:19 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:24:24.408179 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=28 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:24:26.372217 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a4f9a-83d5-4d34-bc5a-38a036bbb98a None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:24:26.373163 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a4f9a-83d5-4d34-bc5a-38a036bbb98a None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:24:26.373163 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a4f9a-83d5-4d34-bc5a-38a036bbb98a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:24:26.390583 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-3f0a4f9a-83d5-4d34-bc5a-38a036bbb98a None None] MFA Rules not processed for user `6bc30b7d536444008e5b5bab45c5c438`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:24:26.391838 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-3f0a4f9a-83d5-4d34-bc5a-38a036bbb98a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:24:26.399338 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1147/2294] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:24:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:24:26.768099 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7df4e61-641c-44e8-846c-3b22d4b76e40 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:24:26.776603 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7df4e61-641c-44e8-846c-3b22d4b76e40 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:24:26.780506 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762f3f51-6759-490d-91bf-0cd9004dff48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:24:26.781187 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f3f51-6759-490d-91bf-0cd9004dff48 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:24:26.781499 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f3f51-6759-490d-91bf-0cd9004dff48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:24:26.781806 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f3f51-6759-490d-91bf-0cd9004dff48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:24:26.783628 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-762f3f51-6759-490d-91bf-0cd9004dff48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:24:26.794085 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762f3f51-6759-490d-91bf-0cd9004dff48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:24:26.795524 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762f3f51-6759-490d-91bf-0cd9004dff48 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:24:26.803739 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1148/2295] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:24:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:24:27.123940 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0a4f9a-83d5-4d34-bc5a-38a036bbb98a None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:24:27.125611 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-3f0a4f9a-83d5-4d34-bc5a-38a036bbb98a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:24:27.209682 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0a4f9a-83d5-4d34-bc5a-38a036bbb98a None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:24:27.214662 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fe215a-1b11-4851-bc5f-6b47d0794ceb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '293c695f15c8437d81a619f840318f44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:24:27.215850 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe215a-1b11-4851-bc5f-6b47d0794ceb service placement] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:24:27.216234 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe215a-1b11-4851-bc5f-6b47d0794ceb service placement] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:24:27.216513 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe215a-1b11-4851-bc5f-6b47d0794ceb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:24:27.218624 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-87fe215a-1b11-4851-bc5f-6b47d0794ceb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:24:27.232275 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fe215a-1b11-4851-bc5f-6b47d0794ceb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:24:27.234077 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fe215a-1b11-4851-bc5f-6b47d0794ceb service placement] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:24:27.247243 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1148/2296] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:24:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:24:27.939381 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e8a200-cd14-42f0-a2da-094fd7c949d1 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:24:27.940012 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e8a200-cd14-42f0-a2da-094fd7c949d1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:24:27.940012 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e8a200-cd14-42f0-a2da-094fd7c949d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:24:27.957903 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-69e8a200-cd14-42f0-a2da-094fd7c949d1 None None] MFA Rules not processed for user `9bcf554f3d6447399e1d9cfe08b0bff7`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:24:27.959125 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-69e8a200-cd14-42f0-a2da-094fd7c949d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:24:28.017249 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1149/2297] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:24:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:24:29.432811 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fe215a-1b11-4851-bc5f-6b47d0794ceb service placement] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:24:29.448188 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fe215a-1b11-4851-bc5f-6b47d0794ceb service placement] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:24:29.457029 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f02f31-e3c3-405b-8032-72fe64af9774 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15039da209ec445d93d49e2bd6d854eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:24:29.457993 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f02f31-e3c3-405b-8032-72fe64af9774 service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:24:29.458286 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f02f31-e3c3-405b-8032-72fe64af9774 service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:24:29.458571 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f02f31-e3c3-405b-8032-72fe64af9774 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:24:29.460508 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-29f02f31-e3c3-405b-8032-72fe64af9774 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:24:29.476365 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f02f31-e3c3-405b-8032-72fe64af9774 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:24:29.478452 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f02f31-e3c3-405b-8032-72fe64af9774 service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:24:29.497701 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1149/2298] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:24:29 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:24:35.761960 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e8a200-cd14-42f0-a2da-094fd7c949d1 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:24:35.763432 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-69e8a200-cd14-42f0-a2da-094fd7c949d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:24:35.775281 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e8a200-cd14-42f0-a2da-094fd7c949d1 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:24:35.778907 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b52cace-4c72-4980-be13-3cb893417f56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:24:35.779548 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b52cace-4c72-4980-be13-3cb893417f56 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:24:35.779836 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b52cace-4c72-4980-be13-3cb893417f56 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:24:35.780115 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b52cace-4c72-4980-be13-3cb893417f56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:24:35.781841 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0b52cace-4c72-4980-be13-3cb893417f56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:24:35.790877 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b52cace-4c72-4980-be13-3cb893417f56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:24:35.792289 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b52cace-4c72-4980-be13-3cb893417f56 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:24:35.800909 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1150/2299] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:24:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:24:39.484925 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=41 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:24:45.797221 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=43 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:24:50.266803 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d345240-a698-4a62-963e-c0fe9f6f4baa None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:24:50.267318 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d345240-a698-4a62-963e-c0fe9f6f4baa None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:24:50.267412 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d345240-a698-4a62-963e-c0fe9f6f4baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:24:50.285861 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-2d345240-a698-4a62-963e-c0fe9f6f4baa None None] MFA Rules not processed for user `78668b939c5847b1b73e9c28b5d8db7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:24:50.287371 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-2d345240-a698-4a62-963e-c0fe9f6f4baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:24:50.296449 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1150/2300] 10.222.0.71 () {58 vars in 1009 bytes} [Mon Jul 4 09:24:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:24:50.348553 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b52cace-4c72-4980-be13-3cb893417f56 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:24:50.362269 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b52cace-4c72-4980-be13-3cb893417f56 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:24:50.367491 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77da9235-2313-4342-8dfb-6a26292c4f02 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88b41b55769a4c449716c3590ed58225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:24:50.368255 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da9235-2313-4342-8dfb-6a26292c4f02 service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:24:50.368544 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da9235-2313-4342-8dfb-6a26292c4f02 service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:24:50.368824 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da9235-2313-4342-8dfb-6a26292c4f02 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:24:50.380647 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77da9235-2313-4342-8dfb-6a26292c4f02 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:24:50.382777 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77da9235-2313-4342-8dfb-6a26292c4f02 service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:24:50.394242 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1151/2301] 10.222.0.71 () {60 vars in 1384 bytes} [Mon Jul 4 09:24:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 04 09:24:53.212084 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d345240-a698-4a62-963e-c0fe9f6f4baa None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:24:53.213554 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-2d345240-a698-4a62-963e-c0fe9f6f4baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:24:53.236332 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d345240-a698-4a62-963e-c0fe9f6f4baa None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:24:53.242841 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7aaacf-a580-4bd4-afa4-81922152e57a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15039da209ec445d93d49e2bd6d854eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:24:53.243508 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7aaacf-a580-4bd4-afa4-81922152e57a service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:24:53.243794 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7aaacf-a580-4bd4-afa4-81922152e57a service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:24:53.244510 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7aaacf-a580-4bd4-afa4-81922152e57a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:24:53.246414 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-9b7aaacf-a580-4bd4-afa4-81922152e57a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:24:53.274942 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7aaacf-a580-4bd4-afa4-81922152e57a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:24:53.277442 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7aaacf-a580-4bd4-afa4-81922152e57a service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:24:53.291285 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1151/2302] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:24:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:25:00.389730 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=6 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:25:03.286359 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=9 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:25:03.672907 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f372936-019b-47bc-b1e9-86d29fce794d None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:03.673768 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f372936-019b-47bc-b1e9-86d29fce794d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:03.673768 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f372936-019b-47bc-b1e9-86d29fce794d None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:03.674790 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1152/2303] 10.222.0.71 () {54 vars in 921 bytes} [Mon Jul 4 09:25:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:25:03.681105 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5403082e-e499-46f8-8a9c-13b9d6f6f96b None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:03.681442 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5403082e-e499-46f8-8a9c-13b9d6f6f96b None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:03.681726 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5403082e-e499-46f8-8a9c-13b9d6f6f96b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:03.727962 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-5403082e-e499-46f8-8a9c-13b9d6f6f96b None None] MFA Rules not processed for user `15039da209ec445d93d49e2bd6d854eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:25:03.729231 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-5403082e-e499-46f8-8a9c-13b9d6f6f96b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:25:03.740838 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1152/2304] 10.222.0.71 () {58 vars in 1018 bytes} [Mon Jul 4 09:25:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:25:03.753086 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f372936-019b-47bc-b1e9-86d29fce794d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:03.765296 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f372936-019b-47bc-b1e9-86d29fce794d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:03.768984 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1cd0b4-5b0d-41a0-8bc3-9b89a35fb9e6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '192724527d774817bc4581474ac848dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:03.769759 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1cd0b4-5b0d-41a0-8bc3-9b89a35fb9e6 service glance] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:03.770116 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1cd0b4-5b0d-41a0-8bc3-9b89a35fb9e6 service glance] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:03.770405 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1cd0b4-5b0d-41a0-8bc3-9b89a35fb9e6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:03.782745 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1cd0b4-5b0d-41a0-8bc3-9b89a35fb9e6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:03.784183 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1cd0b4-5b0d-41a0-8bc3-9b89a35fb9e6 service glance] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:03.797242 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1153/2305] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:25:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:25:03.875115 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f6e34-3008-4638-919c-debe78c8960a None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:03.875409 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f6e34-3008-4638-919c-debe78c8960a None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:03.875693 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f6e34-3008-4638-919c-debe78c8960a None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:03.877249 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1153/2306] 10.222.0.71 () {54 vars in 921 bytes} [Mon Jul 4 09:25:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:25:03.882560 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe8c69-c7b1-4dec-8fe2-c94bb7cefa4b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:03.883533 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe8c69-c7b1-4dec-8fe2-c94bb7cefa4b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:03.884377 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe8c69-c7b1-4dec-8fe2-c94bb7cefa4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:03.904644 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c0fe8c69-c7b1-4dec-8fe2-c94bb7cefa4b None None] MFA Rules not processed for user `15818e58661646f381da0973d4c2ff64`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:25:03.905918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c0fe8c69-c7b1-4dec-8fe2-c94bb7cefa4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:25:03.913767 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1154/2307] 10.222.0.71 () {58 vars in 1018 bytes} [Mon Jul 4 09:25:03 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:25:03.926337 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053f6e34-3008-4638-919c-debe78c8960a None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:03.940711 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053f6e34-3008-4638-919c-debe78c8960a None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:03.945413 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16d2095-a15a-4f24-ad72-fd2c54710527 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:03.946296 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d2095-a15a-4f24-ad72-fd2c54710527 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:03.946817 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d2095-a15a-4f24-ad72-fd2c54710527 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:03.947275 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d2095-a15a-4f24-ad72-fd2c54710527 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:03.957604 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16d2095-a15a-4f24-ad72-fd2c54710527 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:03.959094 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16d2095-a15a-4f24-ad72-fd2c54710527 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:03.969201 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1154/2308] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:25:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:25:04.065141 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ca0f2-bd2e-4aff-b424-dca60964b9b1 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:04.065806 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ca0f2-bd2e-4aff-b424-dca60964b9b1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:04.066566 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ca0f2-bd2e-4aff-b424-dca60964b9b1 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:04.068554 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1155/2309] 10.222.0.71 () {54 vars in 921 bytes} [Mon Jul 4 09:25:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:25:04.076542 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575eb78b-cf79-4dc8-b961-3bdab5a98f2f None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:04.076833 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575eb78b-cf79-4dc8-b961-3bdab5a98f2f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:04.077136 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575eb78b-cf79-4dc8-b961-3bdab5a98f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:04.100236 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-575eb78b-cf79-4dc8-b961-3bdab5a98f2f None None] MFA Rules not processed for user `15039da209ec445d93d49e2bd6d854eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:25:04.101484 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-575eb78b-cf79-4dc8-b961-3bdab5a98f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:25:04.112295 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1155/2310] 10.222.0.71 () {58 vars in 1018 bytes} [Mon Jul 4 09:25:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:25:04.123529 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2ca0f2-bd2e-4aff-b424-dca60964b9b1 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:04.135458 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2ca0f2-bd2e-4aff-b424-dca60964b9b1 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:04.139543 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213759e5-4807-40f5-a6ab-f2ee507487f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:04.140199 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213759e5-4807-40f5-a6ab-f2ee507487f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:04.140482 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213759e5-4807-40f5-a6ab-f2ee507487f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:04.140757 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213759e5-4807-40f5-a6ab-f2ee507487f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:04.152822 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213759e5-4807-40f5-a6ab-f2ee507487f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:04.154835 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213759e5-4807-40f5-a6ab-f2ee507487f6 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:04.166039 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1156/2311] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:25:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:25:05.595800 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575eb78b-cf79-4dc8-b961-3bdab5a98f2f None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:05.607693 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575eb78b-cf79-4dc8-b961-3bdab5a98f2f None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:05.613147 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d54107c-e4ee-4f68-9a76-4340c3e15e80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15039da209ec445d93d49e2bd6d854eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:05.613797 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d54107c-e4ee-4f68-9a76-4340c3e15e80 service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:05.614082 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d54107c-e4ee-4f68-9a76-4340c3e15e80 service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:05.614365 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d54107c-e4ee-4f68-9a76-4340c3e15e80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:05.616151 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-8d54107c-e4ee-4f68-9a76-4340c3e15e80 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:25:05.628269 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d54107c-e4ee-4f68-9a76-4340c3e15e80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:05.629947 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d54107c-e4ee-4f68-9a76-4340c3e15e80 service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:05.640298 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1156/2312] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:25:05 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:25:05.847137 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a9eea-0628-4f42-9695-26b3521c84b7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:05.847624 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a9eea-0628-4f42-9695-26b3521c84b7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:05.847771 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a9eea-0628-4f42-9695-26b3521c84b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:05.869263 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a69a9eea-0628-4f42-9695-26b3521c84b7 None None] MFA Rules not processed for user `78668b939c5847b1b73e9c28b5d8db7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:25:05.870462 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a69a9eea-0628-4f42-9695-26b3521c84b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:25:05.879175 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1157/2313] 10.222.0.71 () {58 vars in 1009 bytes} [Mon Jul 4 09:25:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:25:05.895066 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d54107c-e4ee-4f68-9a76-4340c3e15e80 service nova] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:05.916662 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d54107c-e4ee-4f68-9a76-4340c3e15e80 service nova] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:05.922500 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62d4208-7b08-4904-90d9-c53df9ff0e11 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88b41b55769a4c449716c3590ed58225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:05.923159 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d4208-7b08-4904-90d9-c53df9ff0e11 service swift] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:05.923442 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d4208-7b08-4904-90d9-c53df9ff0e11 service swift] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:05.923773 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d4208-7b08-4904-90d9-c53df9ff0e11 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:05.934487 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62d4208-7b08-4904-90d9-c53df9ff0e11 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:05.936964 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62d4208-7b08-4904-90d9-c53df9ff0e11 service swift] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:05.946758 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1157/2314] 10.222.0.71 () {60 vars in 1384 bytes} [Mon Jul 4 09:25:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 09:25:15.862606 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=18 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:25:15.942462 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=30 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:25:23.346572 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69a9eea-0628-4f42-9695-26b3521c84b7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:23.359737 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69a9eea-0628-4f42-9695-26b3521c84b7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:23.365941 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554698b8-b364-4f14-a3b2-a6d01da353ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15039da209ec445d93d49e2bd6d854eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:23.366703 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554698b8-b364-4f14-a3b2-a6d01da353ee service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:23.367062 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554698b8-b364-4f14-a3b2-a6d01da353ee service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:23.367414 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554698b8-b364-4f14-a3b2-a6d01da353ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:23.369429 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-554698b8-b364-4f14-a3b2-a6d01da353ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:25:23.383293 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554698b8-b364-4f14-a3b2-a6d01da353ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:23.384908 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554698b8-b364-4f14-a3b2-a6d01da353ee service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:23.400523 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1158/2315] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:25:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:25:23.727511 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150430c4-9652-4582-8093-e4aefd1c4a29 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:23.728022 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150430c4-9652-4582-8093-e4aefd1c4a29 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:23.728138 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150430c4-9652-4582-8093-e4aefd1c4a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:23.747101 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-150430c4-9652-4582-8093-e4aefd1c4a29 None None] MFA Rules not processed for user `78668b939c5847b1b73e9c28b5d8db7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:25:23.748844 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-150430c4-9652-4582-8093-e4aefd1c4a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:25:23.756786 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1158/2316] 10.222.0.71 () {58 vars in 967 bytes} [Mon Jul 4 09:25:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:25:23.770315 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554698b8-b364-4f14-a3b2-a6d01da353ee service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:23.781367 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554698b8-b364-4f14-a3b2-a6d01da353ee service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:23.786428 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a456925e-ea91-4bf8-98d2-0d977158028b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88b41b55769a4c449716c3590ed58225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:23.787133 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a456925e-ea91-4bf8-98d2-0d977158028b service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:23.787482 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a456925e-ea91-4bf8-98d2-0d977158028b service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:23.787825 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a456925e-ea91-4bf8-98d2-0d977158028b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:23.796054 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a456925e-ea91-4bf8-98d2-0d977158028b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:23.797716 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a456925e-ea91-4bf8-98d2-0d977158028b service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:23.805952 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1159/2317] 10.222.0.71 () {60 vars in 1384 bytes} [Mon Jul 4 09:25:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 09:25:33.117480 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150430c4-9652-4582-8093-e4aefd1c4a29 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:33.131436 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150430c4-9652-4582-8093-e4aefd1c4a29 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:33.137196 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b1c046-13b5-40bf-9f84-67bd13b9ccd6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15039da209ec445d93d49e2bd6d854eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:33.137847 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b1c046-13b5-40bf-9f84-67bd13b9ccd6 service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:33.138133 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b1c046-13b5-40bf-9f84-67bd13b9ccd6 service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:33.138411 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b1c046-13b5-40bf-9f84-67bd13b9ccd6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:33.140211 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-96b1c046-13b5-40bf-9f84-67bd13b9ccd6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:25:33.150876 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b1c046-13b5-40bf-9f84-67bd13b9ccd6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:33.152456 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b1c046-13b5-40bf-9f84-67bd13b9ccd6 service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:33.161978 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1159/2318] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:25:33 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:25:33.802801 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=12 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:25:37.856470 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a456925e-ea91-4bf8-98d2-0d977158028b service swift] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:37.866403 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a456925e-ea91-4bf8-98d2-0d977158028b service swift] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:37.869946 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615b5d58-d847-4467-bbbc-5154fed217bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:37.870579 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615b5d58-d847-4467-bbbc-5154fed217bb service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:37.870864 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615b5d58-d847-4467-bbbc-5154fed217bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:37.871148 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615b5d58-d847-4467-bbbc-5154fed217bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:37.872878 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-615b5d58-d847-4467-bbbc-5154fed217bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:25:37.935129 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615b5d58-d847-4467-bbbc-5154fed217bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:37.936542 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615b5d58-d847-4467-bbbc-5154fed217bb service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:37.945883 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1160/2319] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:25:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:25:41.265271 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b1c046-13b5-40bf-9f84-67bd13b9ccd6 service nova] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:41.266690 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-96b1c046-13b5-40bf-9f84-67bd13b9ccd6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:25:41.280883 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b1c046-13b5-40bf-9f84-67bd13b9ccd6 service nova] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:41.286201 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e935076-e0d2-470b-bb91-3f950753e48b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '293c695f15c8437d81a619f840318f44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:41.286851 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e935076-e0d2-470b-bb91-3f950753e48b service placement] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:41.287136 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e935076-e0d2-470b-bb91-3f950753e48b service placement] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:41.287410 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e935076-e0d2-470b-bb91-3f950753e48b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:41.289403 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0e935076-e0d2-470b-bb91-3f950753e48b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:25:41.302286 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e935076-e0d2-470b-bb91-3f950753e48b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:41.303934 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e935076-e0d2-470b-bb91-3f950753e48b service placement] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:41.316785 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1160/2320] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:25:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:25:41.325580 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615b5d58-d847-4467-bbbc-5154fed217bb service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:41.329073 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-615b5d58-d847-4467-bbbc-5154fed217bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:25:41.342654 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615b5d58-d847-4467-bbbc-5154fed217bb service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:41.348859 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8274c984-5787-4a60-9c64-42cbaf890621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:41.350072 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8274c984-5787-4a60-9c64-42cbaf890621 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:41.350724 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8274c984-5787-4a60-9c64-42cbaf890621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:41.351669 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8274c984-5787-4a60-9c64-42cbaf890621 admin admin] PATH_INFO: `/v3/users/9bcf554f3d6447399e1d9cfe08b0bff7` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:41.353260 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8274c984-5787-4a60-9c64-42cbaf890621 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bcf554f3d6447399e1d9cfe08b0bff7)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:41.356286 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8274c984-5787-4a60-9c64-42cbaf890621 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:41.432104 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8274c984-5787-4a60-9c64-42cbaf890621 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bcf554f3d6447399e1d9cfe08b0bff7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:41.433024 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8274c984-5787-4a60-9c64-42cbaf890621 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bcf554f3d6447399e1d9cfe08b0bff7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:41.449350 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8274c984-5787-4a60-9c64-42cbaf890621 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bcf554f3d6447399e1d9cfe08b0bff7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:41.462890 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8274c984-5787-4a60-9c64-42cbaf890621 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bcf554f3d6447399e1d9cfe08b0bff7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:41.463570 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8274c984-5787-4a60-9c64-42cbaf890621 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bcf554f3d6447399e1d9cfe08b0bff7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:41.476343 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1161/2321] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:25:41 2022] DELETE /identity/v3/users/9bcf554f3d6447399e1d9cfe08b0bff7 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:25:41.656854 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e935076-e0d2-470b-bb91-3f950753e48b service placement] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:41.668208 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e935076-e0d2-470b-bb91-3f950753e48b service placement] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:41.725719 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06113c3a-3476-4656-a53c-ea02c423c58c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:41.726582 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06113c3a-3476-4656-a53c-ea02c423c58c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:41.726864 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06113c3a-3476-4656-a53c-ea02c423c58c admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:41.727139 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06113c3a-3476-4656-a53c-ea02c423c58c admin admin] PATH_INFO: `/v3/projects/930dc66606e94d9eb82cc50a67d009fb` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:41.728322 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06113c3a-3476-4656-a53c-ea02c423c58c admin admin] RBAC: Authorizing `identity:delete_project(project_id=930dc66606e94d9eb82cc50a67d009fb)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:41.730278 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06113c3a-3476-4656-a53c-ea02c423c58c admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:41.775021 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-06113c3a-3476-4656-a53c-ea02c423c58c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '930dc66606e94d9eb82cc50a67d009fb', 'request_id': 'req-06113c3a-3476-4656-a53c-ea02c423c58c'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:41.780308 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-06113c3a-3476-4656-a53c-ea02c423c58c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '930dc66606e94d9eb82cc50a67d009fb', 'request_id': 'req-06113c3a-3476-4656-a53c-ea02c423c58c'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:41.786525 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-06113c3a-3476-4656-a53c-ea02c423c58c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '930dc66606e94d9eb82cc50a67d009fb', 'request_id': 'req-06113c3a-3476-4656-a53c-ea02c423c58c'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:41.798086 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-06113c3a-3476-4656-a53c-ea02c423c58c admin admin] The token cache is being invalidate because project 930dc66606e94d9eb82cc50a67d009fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:25:41.798990 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-06113c3a-3476-4656-a53c-ea02c423c58c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:41.799536 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-06113c3a-3476-4656-a53c-ea02c423c58c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:41.801083 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1161/2322] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:25:41 2022] DELETE /identity/v3/projects/930dc66606e94d9eb82cc50a67d009fb => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:25:41.806085 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee218dd-5b79-495f-9719-1f46a29d95d7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:41.806425 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee218dd-5b79-495f-9719-1f46a29d95d7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:41.806702 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee218dd-5b79-495f-9719-1f46a29d95d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:41.824871 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-cee218dd-5b79-495f-9719-1f46a29d95d7 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:25:41.826109 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cee218dd-5b79-495f-9719-1f46a29d95d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:25:41.886070 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1162/2323] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:25:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:25:41.893842 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13dbe7-a98f-481c-8163-b57ca5a428ee None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:41.894138 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13dbe7-a98f-481c-8163-b57ca5a428ee None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:41.894426 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13dbe7-a98f-481c-8163-b57ca5a428ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:41.912609 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-5a13dbe7-a98f-481c-8163-b57ca5a428ee None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:25:41.913781 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-5a13dbe7-a98f-481c-8163-b57ca5a428ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:25:41.921684 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1162/2324] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:25:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:25:41.926194 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee218dd-5b79-495f-9719-1f46a29d95d7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:41.935668 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee218dd-5b79-495f-9719-1f46a29d95d7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:41.940911 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af77fe01-d7fc-4c37-8a18-cc91e6f07ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:41.941621 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af77fe01-d7fc-4c37-8a18-cc91e6f07ef5 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:41.941889 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af77fe01-d7fc-4c37-8a18-cc91e6f07ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:41.942166 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af77fe01-d7fc-4c37-8a18-cc91e6f07ef5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:41.942886 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af77fe01-d7fc-4c37-8a18-cc91e6f07ef5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:25:41.943191 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af77fe01-d7fc-4c37-8a18-cc91e6f07ef5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:41.944779 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af77fe01-d7fc-4c37-8a18-cc91e6f07ef5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:41.946256 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:25:41.958004 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1163/2325] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:25:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:25:41.962199 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a13dbe7-a98f-481c-8163-b57ca5a428ee None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:41.972011 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a13dbe7-a98f-481c-8163-b57ca5a428ee None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:41.976789 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0107d2a2-f49e-4691-abd8-cfaa22addb3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:41.977499 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0107d2a2-f49e-4691-abd8-cfaa22addb3e admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:41.977788 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0107d2a2-f49e-4691-abd8-cfaa22addb3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:41.978068 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0107d2a2-f49e-4691-abd8-cfaa22addb3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:41.978770 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0107d2a2-f49e-4691-abd8-cfaa22addb3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:41.980680 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0107d2a2-f49e-4691-abd8-cfaa22addb3e admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:42.018869 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1163/2326] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:25:41 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:25:42.020449 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af77fe01-d7fc-4c37-8a18-cc91e6f07ef5 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:42.030328 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af77fe01-d7fc-4c37-8a18-cc91e6f07ef5 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:42.087270 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b91c4f7-f696-4a62-a9b8-4aa2c55bbdc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:42.087960 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91c4f7-f696-4a62-a9b8-4aa2c55bbdc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:42.088232 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91c4f7-f696-4a62-a9b8-4aa2c55bbdc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:42.088517 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91c4f7-f696-4a62-a9b8-4aa2c55bbdc2 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:42.089253 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b91c4f7-f696-4a62-a9b8-4aa2c55bbdc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:42.091240 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b91c4f7-f696-4a62-a9b8-4aa2c55bbdc2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:42.111362 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1164/2327] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:25:42 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:25:42.115924 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0107d2a2-f49e-4691-abd8-cfaa22addb3e admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:42.127377 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0107d2a2-f49e-4691-abd8-cfaa22addb3e admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:42.132079 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdec117-f5f9-4780-a2e9-70dbca5bb89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:42.132744 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdec117-f5f9-4780-a2e9-70dbca5bb89d admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:42.133032 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdec117-f5f9-4780-a2e9-70dbca5bb89d admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:42.133339 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdec117-f5f9-4780-a2e9-70dbca5bb89d admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:42.134071 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdec117-f5f9-4780-a2e9-70dbca5bb89d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:42.135689 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdec117-f5f9-4780-a2e9-70dbca5bb89d admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:42.145733 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1164/2328] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:25:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:25:42.149404 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b91c4f7-f696-4a62-a9b8-4aa2c55bbdc2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:42.159845 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b91c4f7-f696-4a62-a9b8-4aa2c55bbdc2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:42.164533 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd02b6e1-038a-4e2a-8ea1-94be1cb7c23e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:42.165161 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02b6e1-038a-4e2a-8ea1-94be1cb7c23e admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:42.165452 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02b6e1-038a-4e2a-8ea1-94be1cb7c23e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:42.165733 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02b6e1-038a-4e2a-8ea1-94be1cb7c23e admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:42.166450 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd02b6e1-038a-4e2a-8ea1-94be1cb7c23e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:42.168037 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd02b6e1-038a-4e2a-8ea1-94be1cb7c23e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:42.179151 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1165/2329] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:25:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:25:42.183018 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdec117-f5f9-4780-a2e9-70dbca5bb89d admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:42.194045 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdec117-f5f9-4780-a2e9-70dbca5bb89d admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:42.200059 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05befb1f-90bd-4eb9-83d2-3a9546198a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:42.201271 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05befb1f-90bd-4eb9-83d2-3a9546198a5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:42.201713 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05befb1f-90bd-4eb9-83d2-3a9546198a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:42.201986 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05befb1f-90bd-4eb9-83d2-3a9546198a5a admin admin] PATH_INFO: `/v3/projects/6df585d8538848e2a82604efefb49dc9/users/86c3e25fa55b4719bdebf7397dea65d6/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:42.208343 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:25:42.227886 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05befb1f-90bd-4eb9-83d2-3a9546198a5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6df585d8538848e2a82604efefb49dc9, user_id=86c3e25fa55b4719bdebf7397dea65d6, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:42.231562 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05befb1f-90bd-4eb9-83d2-3a9546198a5a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:42.246425 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1165/2330] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:25:42 2022] PUT /identity/v3/projects/6df585d8538848e2a82604efefb49dc9/users/86c3e25fa55b4719bdebf7397dea65d6/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:25:42.256690 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f2e38-eda9-4c1d-b1a6-2f4e499473fe None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:42.256987 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f2e38-eda9-4c1d-b1a6-2f4e499473fe None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:42.257317 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f2e38-eda9-4c1d-b1a6-2f4e499473fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:42.284106 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-fe3f2e38-eda9-4c1d-b1a6-2f4e499473fe None None] MFA Rules not processed for user `86c3e25fa55b4719bdebf7397dea65d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:25:42.285424 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fe3f2e38-eda9-4c1d-b1a6-2f4e499473fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:25:42.353677 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1166/2331] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:25:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:25:42.365809 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05befb1f-90bd-4eb9-83d2-3a9546198a5a admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:42.367573 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-05befb1f-90bd-4eb9-83d2-3a9546198a5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:25:42.442115 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05befb1f-90bd-4eb9-83d2-3a9546198a5a admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:42.447530 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67155ecd-3b80-4126-8eec-209560b7ad48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15039da209ec445d93d49e2bd6d854eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:42.448181 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67155ecd-3b80-4126-8eec-209560b7ad48 service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:42.448466 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67155ecd-3b80-4126-8eec-209560b7ad48 service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:42.448745 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67155ecd-3b80-4126-8eec-209560b7ad48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:42.457972 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67155ecd-3b80-4126-8eec-209560b7ad48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:42.459577 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67155ecd-3b80-4126-8eec-209560b7ad48 service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:42.468939 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1166/2332] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:25:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:25:49.763157 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3f2e38-eda9-4c1d-b1a6-2f4e499473fe None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:49.825947 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3f2e38-eda9-4c1d-b1a6-2f4e499473fe None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:49.830453 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ebdfeb-09bc-4540-a3a5-37e35d7e6e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:49.832855 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ebdfeb-09bc-4540-a3a5-37e35d7e6e91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:49.833169 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ebdfeb-09bc-4540-a3a5-37e35d7e6e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:49.833448 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ebdfeb-09bc-4540-a3a5-37e35d7e6e91 admin admin] PATH_INFO: `/v3/users/86c3e25fa55b4719bdebf7397dea65d6` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:49.834655 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ebdfeb-09bc-4540-a3a5-37e35d7e6e91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86c3e25fa55b4719bdebf7397dea65d6)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:49.836622 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ebdfeb-09bc-4540-a3a5-37e35d7e6e91 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:49.839116 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:25:49.890169 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-56ebdfeb-09bc-4540-a3a5-37e35d7e6e91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86c3e25fa55b4719bdebf7397dea65d6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:49.890773 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-56ebdfeb-09bc-4540-a3a5-37e35d7e6e91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86c3e25fa55b4719bdebf7397dea65d6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:49.897593 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-56ebdfeb-09bc-4540-a3a5-37e35d7e6e91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86c3e25fa55b4719bdebf7397dea65d6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:49.910028 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-56ebdfeb-09bc-4540-a3a5-37e35d7e6e91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86c3e25fa55b4719bdebf7397dea65d6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:49.910648 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-56ebdfeb-09bc-4540-a3a5-37e35d7e6e91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86c3e25fa55b4719bdebf7397dea65d6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:49.922574 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1167/2333] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:25:49 2022] DELETE /identity/v3/users/86c3e25fa55b4719bdebf7397dea65d6 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:25:49.927687 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e778d98-77a3-4645-8561-8fb399e0e27e None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:49.928241 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e778d98-77a3-4645-8561-8fb399e0e27e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:49.928349 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e778d98-77a3-4645-8561-8fb399e0e27e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:49.945871 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-4e778d98-77a3-4645-8561-8fb399e0e27e None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:25:49.947061 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4e778d98-77a3-4645-8561-8fb399e0e27e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:25:50.004581 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1167/2334] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:25:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:25:50.013841 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ebdfeb-09bc-4540-a3a5-37e35d7e6e91 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:50.015303 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-56ebdfeb-09bc-4540-a3a5-37e35d7e6e91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:25:50.072394 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ebdfeb-09bc-4540-a3a5-37e35d7e6e91 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:50.076053 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b71d5e7-62e0-40c9-8b0e-ca2b1874de2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:50.076699 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b71d5e7-62e0-40c9-8b0e-ca2b1874de2b service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:50.076991 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b71d5e7-62e0-40c9-8b0e-ca2b1874de2b service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:50.077306 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b71d5e7-62e0-40c9-8b0e-ca2b1874de2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:50.087125 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b71d5e7-62e0-40c9-8b0e-ca2b1874de2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:50.088554 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b71d5e7-62e0-40c9-8b0e-ca2b1874de2b service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:50.098918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1168/2335] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:25:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:25:50.368463 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e778d98-77a3-4645-8561-8fb399e0e27e None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:50.369926 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4e778d98-77a3-4645-8561-8fb399e0e27e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:25:50.385976 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e778d98-77a3-4645-8561-8fb399e0e27e None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:50.392198 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0807884d-b668-49a2-b542-cdc93b619aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:50.394561 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0807884d-b668-49a2-b542-cdc93b619aae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:50.394561 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0807884d-b668-49a2-b542-cdc93b619aae admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:50.394561 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0807884d-b668-49a2-b542-cdc93b619aae admin admin] PATH_INFO: `/v3/projects/6df585d8538848e2a82604efefb49dc9` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:50.395886 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0807884d-b668-49a2-b542-cdc93b619aae admin admin] RBAC: Authorizing `identity:delete_project(project_id=6df585d8538848e2a82604efefb49dc9)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:50.397848 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0807884d-b668-49a2-b542-cdc93b619aae admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:50.448737 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0807884d-b668-49a2-b542-cdc93b619aae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6df585d8538848e2a82604efefb49dc9', 'request_id': 'req-0807884d-b668-49a2-b542-cdc93b619aae'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:50.454135 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0807884d-b668-49a2-b542-cdc93b619aae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6df585d8538848e2a82604efefb49dc9', 'request_id': 'req-0807884d-b668-49a2-b542-cdc93b619aae'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:50.461069 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0807884d-b668-49a2-b542-cdc93b619aae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6df585d8538848e2a82604efefb49dc9', 'request_id': 'req-0807884d-b668-49a2-b542-cdc93b619aae'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:50.464058 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:25:50.477374 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0807884d-b668-49a2-b542-cdc93b619aae admin admin] The token cache is being invalidate because project 6df585d8538848e2a82604efefb49dc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:25:50.477672 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0807884d-b668-49a2-b542-cdc93b619aae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:50.478312 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0807884d-b668-49a2-b542-cdc93b619aae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:50.479880 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1168/2336] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:25:50 2022] DELETE /identity/v3/projects/6df585d8538848e2a82604efefb49dc9 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:25:50.484903 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3973867-c703-429b-ae1c-d58cc1cbda88 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:50.485303 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3973867-c703-429b-ae1c-d58cc1cbda88 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:50.485687 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3973867-c703-429b-ae1c-d58cc1cbda88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:50.503727 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a3973867-c703-429b-ae1c-d58cc1cbda88 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:25:50.504953 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a3973867-c703-429b-ae1c-d58cc1cbda88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:25:50.572927 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1169/2337] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:25:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:25:50.582245 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a196f-6428-4a0a-b508-e599eec89032 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:50.582540 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a196f-6428-4a0a-b508-e599eec89032 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:50.582822 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a196f-6428-4a0a-b508-e599eec89032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:50.601148 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-1c7a196f-6428-4a0a-b508-e599eec89032 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:25:50.602361 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-1c7a196f-6428-4a0a-b508-e599eec89032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:25:50.610326 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1169/2338] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:25:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:25:50.614639 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3973867-c703-429b-ae1c-d58cc1cbda88 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:50.624295 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3973867-c703-429b-ae1c-d58cc1cbda88 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:50.629770 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a839e4ed-eb3e-4086-9b87-ab567c703f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:50.630424 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a839e4ed-eb3e-4086-9b87-ab567c703f33 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:50.630718 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a839e4ed-eb3e-4086-9b87-ab567c703f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:50.631025 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a839e4ed-eb3e-4086-9b87-ab567c703f33 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:50.631766 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a839e4ed-eb3e-4086-9b87-ab567c703f33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:25:50.632066 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a839e4ed-eb3e-4086-9b87-ab567c703f33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:50.633685 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a839e4ed-eb3e-4086-9b87-ab567c703f33 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:50.645832 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1170/2339] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:25:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:25:50.650137 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7a196f-6428-4a0a-b508-e599eec89032 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:50.659878 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7a196f-6428-4a0a-b508-e599eec89032 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:50.664572 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d8e1c2-1f01-4608-9329-2a0f7a1a776f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:50.665284 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d8e1c2-1f01-4608-9329-2a0f7a1a776f admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:50.665573 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d8e1c2-1f01-4608-9329-2a0f7a1a776f admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:50.665852 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d8e1c2-1f01-4608-9329-2a0f7a1a776f admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:50.666597 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d8e1c2-1f01-4608-9329-2a0f7a1a776f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:50.668544 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d8e1c2-1f01-4608-9329-2a0f7a1a776f admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:50.695381 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1170/2340] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:25:50 2022] POST /identity/v3/projects => generated 403 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:25:50.699580 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a839e4ed-eb3e-4086-9b87-ab567c703f33 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:50.710164 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a839e4ed-eb3e-4086-9b87-ab567c703f33 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:50.717859 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:25:50.768831 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4254ba36-9b69-4eed-8528-25eb33ee12ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:50.769531 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4254ba36-9b69-4eed-8528-25eb33ee12ff admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:50.769817 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4254ba36-9b69-4eed-8528-25eb33ee12ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:50.770095 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4254ba36-9b69-4eed-8528-25eb33ee12ff admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:50.770783 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4254ba36-9b69-4eed-8528-25eb33ee12ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:50.772675 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4254ba36-9b69-4eed-8528-25eb33ee12ff admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:50.792211 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1171/2341] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:25:50 2022] POST /identity/v3/users => generated 302 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:25:50.796501 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d8e1c2-1f01-4608-9329-2a0f7a1a776f admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:50.807752 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d8e1c2-1f01-4608-9329-2a0f7a1a776f admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:50.812324 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d04d61-b92a-4c67-a73a-46fa88228bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:50.812974 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d04d61-b92a-4c67-a73a-46fa88228bb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:50.813285 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d04d61-b92a-4c67-a73a-46fa88228bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:50.813562 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d04d61-b92a-4c67-a73a-46fa88228bb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:50.814304 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d04d61-b92a-4c67-a73a-46fa88228bb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:50.816930 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d04d61-b92a-4c67-a73a-46fa88228bb5 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:50.827272 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1171/2342] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:25:50.831524 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4254ba36-9b69-4eed-8528-25eb33ee12ff admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:50.841476 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4254ba36-9b69-4eed-8528-25eb33ee12ff admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:50.846423 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f74c769-0f79-4196-a697-9f72be60a8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:50.847077 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f74c769-0f79-4196-a697-9f72be60a8d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:50.847363 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f74c769-0f79-4196-a697-9f72be60a8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:50.847643 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f74c769-0f79-4196-a697-9f72be60a8d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:50.848367 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f74c769-0f79-4196-a697-9f72be60a8d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:50.849978 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f74c769-0f79-4196-a697-9f72be60a8d2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:50.863747 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1172/2343] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:25:50.864533 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d04d61-b92a-4c67-a73a-46fa88228bb5 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:50.874763 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d04d61-b92a-4c67-a73a-46fa88228bb5 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:50.880259 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c1e399-cddf-42eb-9723-57c9f2f468a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:50.881170 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1e399-cddf-42eb-9723-57c9f2f468a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:50.881453 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1e399-cddf-42eb-9723-57c9f2f468a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:50.881729 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1e399-cddf-42eb-9723-57c9f2f468a2 admin admin] PATH_INFO: `/v3/projects/996e6c0a5d15420783eb94290cf923ba/users/b41c004947a142dbaa55770ca2aafe67/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:50.896751 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c1e399-cddf-42eb-9723-57c9f2f468a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=996e6c0a5d15420783eb94290cf923ba, user_id=b41c004947a142dbaa55770ca2aafe67, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:50.899719 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c1e399-cddf-42eb-9723-57c9f2f468a2 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:50.913003 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1172/2344] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:25:50 2022] PUT /identity/v3/projects/996e6c0a5d15420783eb94290cf923ba/users/b41c004947a142dbaa55770ca2aafe67/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:25:50.923366 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0183cdb5-55b6-40a1-a3de-bf0691d84f33 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:50.923659 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0183cdb5-55b6-40a1-a3de-bf0691d84f33 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:50.923948 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0183cdb5-55b6-40a1-a3de-bf0691d84f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:50.953153 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0183cdb5-55b6-40a1-a3de-bf0691d84f33 None None] MFA Rules not processed for user `b41c004947a142dbaa55770ca2aafe67`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:25:50.954312 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0183cdb5-55b6-40a1-a3de-bf0691d84f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:25:51.020003 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1173/2345] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:25:50 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:25:51.024392 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c1e399-cddf-42eb-9723-57c9f2f468a2 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:51.085751 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c1e399-cddf-42eb-9723-57c9f2f468a2 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:51.090383 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85093077-d747-41ce-a34c-9dd927ceb2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:51.091037 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85093077-d747-41ce-a34c-9dd927ceb2b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:51.091320 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85093077-d747-41ce-a34c-9dd927ceb2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:51.091596 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85093077-d747-41ce-a34c-9dd927ceb2b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:51.092274 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85093077-d747-41ce-a34c-9dd927ceb2b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:51.094451 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85093077-d747-41ce-a34c-9dd927ceb2b6 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:51.119554 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1173/2346] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:25:51 2022] POST /identity/v3/projects => generated 403 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:25:51.123676 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0183cdb5-55b6-40a1-a3de-bf0691d84f33 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:51.135333 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0183cdb5-55b6-40a1-a3de-bf0691d84f33 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:51.191149 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225918a8-7609-4f24-8f33-8ee6d94e5894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:51.191818 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225918a8-7609-4f24-8f33-8ee6d94e5894 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:51.192107 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225918a8-7609-4f24-8f33-8ee6d94e5894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:51.192388 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225918a8-7609-4f24-8f33-8ee6d94e5894 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:51.193113 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225918a8-7609-4f24-8f33-8ee6d94e5894 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:51.195023 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225918a8-7609-4f24-8f33-8ee6d94e5894 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:51.214522 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1174/2347] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:25:51 2022] POST /identity/v3/users => generated 308 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:25:51.218776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85093077-d747-41ce-a34c-9dd927ceb2b6 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:51.229741 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85093077-d747-41ce-a34c-9dd927ceb2b6 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:51.234513 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9275ecdf-aa48-4930-a9fc-dc00fdfb8a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:51.235167 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275ecdf-aa48-4930-a9fc-dc00fdfb8a78 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:51.235451 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275ecdf-aa48-4930-a9fc-dc00fdfb8a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:51.235753 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275ecdf-aa48-4930-a9fc-dc00fdfb8a78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:51.236476 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9275ecdf-aa48-4930-a9fc-dc00fdfb8a78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:51.238674 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9275ecdf-aa48-4930-a9fc-dc00fdfb8a78 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:51.249137 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1174/2348] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:25:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:25:51.253068 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225918a8-7609-4f24-8f33-8ee6d94e5894 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:51.262912 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225918a8-7609-4f24-8f33-8ee6d94e5894 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:51.267494 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c70b8da-3222-48e9-8c1a-e7146bd1b479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:51.268543 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c70b8da-3222-48e9-8c1a-e7146bd1b479 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:51.268826 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c70b8da-3222-48e9-8c1a-e7146bd1b479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:51.269136 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c70b8da-3222-48e9-8c1a-e7146bd1b479 admin admin] PATH_INFO: `/v3/projects/fad10ed8f4d94e0eb20e34c2c68aa70b/users/85621b96f6a8498abedbd5ee1f294b8f/roles/f36e053b442540f08041ac7878760f65` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:51.272118 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:25:51.286453 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c70b8da-3222-48e9-8c1a-e7146bd1b479 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fad10ed8f4d94e0eb20e34c2c68aa70b, user_id=85621b96f6a8498abedbd5ee1f294b8f, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:51.289276 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c70b8da-3222-48e9-8c1a-e7146bd1b479 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:51.302853 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1175/2349] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:25:51 2022] PUT /identity/v3/projects/fad10ed8f4d94e0eb20e34c2c68aa70b/users/85621b96f6a8498abedbd5ee1f294b8f/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:25:51.306723 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9275ecdf-aa48-4930-a9fc-dc00fdfb8a78 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:51.316528 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9275ecdf-aa48-4930-a9fc-dc00fdfb8a78 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:51.373276 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dae989a-93c9-4117-927c-76c4d781f153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:51.373938 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dae989a-93c9-4117-927c-76c4d781f153 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:51.374222 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dae989a-93c9-4117-927c-76c4d781f153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:51.374497 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dae989a-93c9-4117-927c-76c4d781f153 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:51.375210 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dae989a-93c9-4117-927c-76c4d781f153 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:51.376798 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dae989a-93c9-4117-927c-76c4d781f153 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:51.382873 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:25:51.392595 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c70b8da-3222-48e9-8c1a-e7146bd1b479 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:51.394998 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1175/2350] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:25:51 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:25:51.404973 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c70b8da-3222-48e9-8c1a-e7146bd1b479 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:51.409871 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af29a93-58ae-4b73-a4c0-866055c43116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:51.410787 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af29a93-58ae-4b73-a4c0-866055c43116 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:51.411073 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af29a93-58ae-4b73-a4c0-866055c43116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:51.411351 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af29a93-58ae-4b73-a4c0-866055c43116 admin admin] PATH_INFO: `/v3/projects/fad10ed8f4d94e0eb20e34c2c68aa70b/users/85621b96f6a8498abedbd5ee1f294b8f/roles/f36e053b442540f08041ac7878760f65` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:51.413343 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af29a93-58ae-4b73-a4c0-866055c43116 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fad10ed8f4d94e0eb20e34c2c68aa70b, user_id=85621b96f6a8498abedbd5ee1f294b8f, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:51.416013 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af29a93-58ae-4b73-a4c0-866055c43116 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:51.426440 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1176/2351] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:25:51 2022] PUT /identity/v3/projects/fad10ed8f4d94e0eb20e34c2c68aa70b/users/85621b96f6a8498abedbd5ee1f294b8f/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:25:51.437696 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82522e9-1d9d-47b4-ba22-7a870c3de474 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:51.437995 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82522e9-1d9d-47b4-ba22-7a870c3de474 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:51.438282 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82522e9-1d9d-47b4-ba22-7a870c3de474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:51.465992 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-a82522e9-1d9d-47b4-ba22-7a870c3de474 None None] MFA Rules not processed for user `85621b96f6a8498abedbd5ee1f294b8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:25:51.467142 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a82522e9-1d9d-47b4-ba22-7a870c3de474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:25:51.538228 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1176/2352] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:25:51 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:25:51.549004 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af29a93-58ae-4b73-a4c0-866055c43116 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:51.551859 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8af29a93-58ae-4b73-a4c0-866055c43116 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:25:51.614625 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af29a93-58ae-4b73-a4c0-866055c43116 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:51.618438 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a626e4-3fbe-47fb-bfc0-f5526b0c14fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:51.619180 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a626e4-3fbe-47fb-bfc0-f5526b0c14fd service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:51.619535 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a626e4-3fbe-47fb-bfc0-f5526b0c14fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:51.619888 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a626e4-3fbe-47fb-bfc0-f5526b0c14fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:51.631791 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a626e4-3fbe-47fb-bfc0-f5526b0c14fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:51.633402 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a626e4-3fbe-47fb-bfc0-f5526b0c14fd service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:51.644881 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1177/2353] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:25:51 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:25:51.797613 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82522e9-1d9d-47b4-ba22-7a870c3de474 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:51.863794 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82522e9-1d9d-47b4-ba22-7a870c3de474 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:51.868406 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2450944e-10fd-4657-ad86-120a8d571ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:51.869314 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2450944e-10fd-4657-ad86-120a8d571ae8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:51.869606 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2450944e-10fd-4657-ad86-120a8d571ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:51.869933 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2450944e-10fd-4657-ad86-120a8d571ae8 admin admin] PATH_INFO: `/v3/users/b41c004947a142dbaa55770ca2aafe67` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:51.871151 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2450944e-10fd-4657-ad86-120a8d571ae8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b41c004947a142dbaa55770ca2aafe67)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:51.873137 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2450944e-10fd-4657-ad86-120a8d571ae8 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:51.921738 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-2450944e-10fd-4657-ad86-120a8d571ae8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b41c004947a142dbaa55770ca2aafe67'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:51.930305 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-2450944e-10fd-4657-ad86-120a8d571ae8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b41c004947a142dbaa55770ca2aafe67'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:51.931064 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-2450944e-10fd-4657-ad86-120a8d571ae8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b41c004947a142dbaa55770ca2aafe67'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:51.938570 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-2450944e-10fd-4657-ad86-120a8d571ae8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b41c004947a142dbaa55770ca2aafe67'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:51.951126 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-2450944e-10fd-4657-ad86-120a8d571ae8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b41c004947a142dbaa55770ca2aafe67'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:51.961528 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1177/2354] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:25:51 2022] DELETE /identity/v3/users/b41c004947a142dbaa55770ca2aafe67 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:25:51.962774 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b043a8-099a-4d57-b2cd-6232a4e92db8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:51.963257 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b043a8-099a-4d57-b2cd-6232a4e92db8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:51.964649 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b043a8-099a-4d57-b2cd-6232a4e92db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:51.982665 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-65b043a8-099a-4d57-b2cd-6232a4e92db8 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:25:51.983789 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-65b043a8-099a-4d57-b2cd-6232a4e92db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:25:52.049574 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1178/2355] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:25:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:25:52.060201 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2450944e-10fd-4657-ad86-120a8d571ae8 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:52.062326 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-2450944e-10fd-4657-ad86-120a8d571ae8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:25:52.070535 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:25:52.140802 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2450944e-10fd-4657-ad86-120a8d571ae8 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:52.145831 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a0446a-53f2-4796-9c95-d3452d772ff9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:52.146622 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a0446a-53f2-4796-9c95-d3452d772ff9 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:52.147016 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a0446a-53f2-4796-9c95-d3452d772ff9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:52.147403 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a0446a-53f2-4796-9c95-d3452d772ff9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:52.158331 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a0446a-53f2-4796-9c95-d3452d772ff9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:52.160102 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a0446a-53f2-4796-9c95-d3452d772ff9 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:52.171203 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1178/2356] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:25:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:25:52.467454 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b043a8-099a-4d57-b2cd-6232a4e92db8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:52.469399 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-65b043a8-099a-4d57-b2cd-6232a4e92db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:25:52.505944 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b043a8-099a-4d57-b2cd-6232a4e92db8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:52.515080 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7c633e-1dc0-4de8-a544-4a5e7c37040f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:52.516643 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c633e-1dc0-4de8-a544-4a5e7c37040f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:52.517126 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c633e-1dc0-4de8-a544-4a5e7c37040f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:52.517458 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c633e-1dc0-4de8-a544-4a5e7c37040f admin admin] PATH_INFO: `/v3/projects/996e6c0a5d15420783eb94290cf923ba` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:52.520022 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7c633e-1dc0-4de8-a544-4a5e7c37040f admin admin] RBAC: Authorizing `identity:delete_project(project_id=996e6c0a5d15420783eb94290cf923ba)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:52.521145 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7c633e-1dc0-4de8-a544-4a5e7c37040f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:52.575690 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6f7c633e-1dc0-4de8-a544-4a5e7c37040f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '996e6c0a5d15420783eb94290cf923ba', 'request_id': 'req-6f7c633e-1dc0-4de8-a544-4a5e7c37040f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:52.581739 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6f7c633e-1dc0-4de8-a544-4a5e7c37040f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '996e6c0a5d15420783eb94290cf923ba', 'request_id': 'req-6f7c633e-1dc0-4de8-a544-4a5e7c37040f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:52.588764 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6f7c633e-1dc0-4de8-a544-4a5e7c37040f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '996e6c0a5d15420783eb94290cf923ba', 'request_id': 'req-6f7c633e-1dc0-4de8-a544-4a5e7c37040f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:52.598818 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:25:52.604323 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6f7c633e-1dc0-4de8-a544-4a5e7c37040f admin admin] The token cache is being invalidate because project 996e6c0a5d15420783eb94290cf923ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:25:52.604725 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6f7c633e-1dc0-4de8-a544-4a5e7c37040f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:52.605546 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6f7c633e-1dc0-4de8-a544-4a5e7c37040f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:52.608211 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1179/2357] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:25:52 2022] DELETE /identity/v3/projects/996e6c0a5d15420783eb94290cf923ba => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:25:52.613085 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a0446a-53f2-4796-9c95-d3452d772ff9 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:52.622999 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a0446a-53f2-4796-9c95-d3452d772ff9 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:52.694992 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7839f044-27c3-4673-adc2-13f57f76bdc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:52.695881 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7839f044-27c3-4673-adc2-13f57f76bdc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:52.696171 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7839f044-27c3-4673-adc2-13f57f76bdc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:52.696453 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7839f044-27c3-4673-adc2-13f57f76bdc0 admin admin] PATH_INFO: `/v3/users/85621b96f6a8498abedbd5ee1f294b8f` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:52.697686 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7839f044-27c3-4673-adc2-13f57f76bdc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85621b96f6a8498abedbd5ee1f294b8f)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:52.699638 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7839f044-27c3-4673-adc2-13f57f76bdc0 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:52.749085 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-7839f044-27c3-4673-adc2-13f57f76bdc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85621b96f6a8498abedbd5ee1f294b8f'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:52.756967 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-7839f044-27c3-4673-adc2-13f57f76bdc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85621b96f6a8498abedbd5ee1f294b8f'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:52.757716 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-7839f044-27c3-4673-adc2-13f57f76bdc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85621b96f6a8498abedbd5ee1f294b8f'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:52.765345 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-7839f044-27c3-4673-adc2-13f57f76bdc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85621b96f6a8498abedbd5ee1f294b8f'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:52.777278 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-7839f044-27c3-4673-adc2-13f57f76bdc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85621b96f6a8498abedbd5ee1f294b8f'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:52.786381 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1179/2358] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:25:52 2022] DELETE /identity/v3/users/85621b96f6a8498abedbd5ee1f294b8f => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:25:53.320631 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7c633e-1dc0-4de8-a544-4a5e7c37040f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:53.322467 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6f7c633e-1dc0-4de8-a544-4a5e7c37040f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:25:53.338461 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7c633e-1dc0-4de8-a544-4a5e7c37040f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:53.431529 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61112c2d-43ac-429c-8337-e1dbe3bf686c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:53.432445 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61112c2d-43ac-429c-8337-e1dbe3bf686c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:53.432731 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61112c2d-43ac-429c-8337-e1dbe3bf686c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:53.433010 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61112c2d-43ac-429c-8337-e1dbe3bf686c admin admin] PATH_INFO: `/v3/projects/fad10ed8f4d94e0eb20e34c2c68aa70b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:53.434276 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61112c2d-43ac-429c-8337-e1dbe3bf686c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fad10ed8f4d94e0eb20e34c2c68aa70b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:53.436447 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61112c2d-43ac-429c-8337-e1dbe3bf686c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:53.490089 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-61112c2d-43ac-429c-8337-e1dbe3bf686c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fad10ed8f4d94e0eb20e34c2c68aa70b', 'request_id': 'req-61112c2d-43ac-429c-8337-e1dbe3bf686c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:53.508826 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-61112c2d-43ac-429c-8337-e1dbe3bf686c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fad10ed8f4d94e0eb20e34c2c68aa70b', 'request_id': 'req-61112c2d-43ac-429c-8337-e1dbe3bf686c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:53.530143 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-61112c2d-43ac-429c-8337-e1dbe3bf686c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fad10ed8f4d94e0eb20e34c2c68aa70b', 'request_id': 'req-61112c2d-43ac-429c-8337-e1dbe3bf686c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:53.554328 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-61112c2d-43ac-429c-8337-e1dbe3bf686c admin admin] The token cache is being invalidate because project fad10ed8f4d94e0eb20e34c2c68aa70b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:25:53.554676 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-61112c2d-43ac-429c-8337-e1dbe3bf686c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:53.555453 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-61112c2d-43ac-429c-8337-e1dbe3bf686c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:53.557302 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1180/2359] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:25:53 2022] DELETE /identity/v3/projects/fad10ed8f4d94e0eb20e34c2c68aa70b => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:25:53.562176 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c08371-51e1-4107-98ba-8e3dcc30448e None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:53.562675 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c08371-51e1-4107-98ba-8e3dcc30448e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:53.562933 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c08371-51e1-4107-98ba-8e3dcc30448e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:53.586131 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-e6c08371-51e1-4107-98ba-8e3dcc30448e None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:25:53.587400 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e6c08371-51e1-4107-98ba-8e3dcc30448e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:25:53.612605 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:25:53.666598 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1180/2360] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:25:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:25:53.669174 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7299b47c-0c3b-4171-9a41-efe2402d457f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:53.670464 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7299b47c-0c3b-4171-9a41-efe2402d457f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:53.670973 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7299b47c-0c3b-4171-9a41-efe2402d457f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:53.697156 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7299b47c-0c3b-4171-9a41-efe2402d457f None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:25:53.699414 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7299b47c-0c3b-4171-9a41-efe2402d457f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:25:53.714566 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1181/2361] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:25:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:25:53.716329 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c08371-51e1-4107-98ba-8e3dcc30448e None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:53.727851 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c08371-51e1-4107-98ba-8e3dcc30448e None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:53.733826 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff314a6-507b-41d9-9ed1-86dbf7dd177e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:53.734513 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff314a6-507b-41d9-9ed1-86dbf7dd177e admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:53.734799 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff314a6-507b-41d9-9ed1-86dbf7dd177e admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:53.735080 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff314a6-507b-41d9-9ed1-86dbf7dd177e admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:53.735829 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff314a6-507b-41d9-9ed1-86dbf7dd177e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:25:53.736142 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff314a6-507b-41d9-9ed1-86dbf7dd177e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:53.738862 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff314a6-507b-41d9-9ed1-86dbf7dd177e admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:53.751616 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1181/2362] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:25:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:25:53.757109 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7299b47c-0c3b-4171-9a41-efe2402d457f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:53.771780 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7299b47c-0c3b-4171-9a41-efe2402d457f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:53.777078 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a8d225-34f0-48de-9f28-6542130c5661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:53.777505 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a8d225-34f0-48de-9f28-6542130c5661 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:53.777794 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a8d225-34f0-48de-9f28-6542130c5661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:53.778076 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a8d225-34f0-48de-9f28-6542130c5661 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:53.778779 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a8d225-34f0-48de-9f28-6542130c5661 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:53.781202 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a8d225-34f0-48de-9f28-6542130c5661 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:53.823053 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1182/2363] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:25:53 2022] POST /identity/v3/projects => generated 375 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:25:53.827878 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff314a6-507b-41d9-9ed1-86dbf7dd177e admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:53.855286 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff314a6-507b-41d9-9ed1-86dbf7dd177e admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:53.966487 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377296a8-3a3c-478d-956c-d3800a63a823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:53.973374 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377296a8-3a3c-478d-956c-d3800a63a823 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:53.974108 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377296a8-3a3c-478d-956c-d3800a63a823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:53.974547 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377296a8-3a3c-478d-956c-d3800a63a823 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:53.975597 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377296a8-3a3c-478d-956c-d3800a63a823 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:53.978483 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377296a8-3a3c-478d-956c-d3800a63a823 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:54.016830 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1182/2364] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:25:53 2022] POST /identity/v3/users => generated 288 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:25:54.020895 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a8d225-34f0-48de-9f28-6542130c5661 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:54.037422 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a8d225-34f0-48de-9f28-6542130c5661 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:54.042899 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfa99cf-d3cd-4c13-babe-96b2ee01f135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:54.043594 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa99cf-d3cd-4c13-babe-96b2ee01f135 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:54.043883 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa99cf-d3cd-4c13-babe-96b2ee01f135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:54.044167 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa99cf-d3cd-4c13-babe-96b2ee01f135 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:54.044904 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfa99cf-d3cd-4c13-babe-96b2ee01f135 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:54.046740 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfa99cf-d3cd-4c13-babe-96b2ee01f135 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:54.067799 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1183/2365] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:25:54 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:25:54.074234 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377296a8-3a3c-478d-956c-d3800a63a823 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:54.086115 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377296a8-3a3c-478d-956c-d3800a63a823 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:54.091053 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efabfa7a-c497-4ae1-a44c-cfb86515973d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:54.091741 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efabfa7a-c497-4ae1-a44c-cfb86515973d admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:54.092034 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efabfa7a-c497-4ae1-a44c-cfb86515973d admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:54.092343 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efabfa7a-c497-4ae1-a44c-cfb86515973d admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:54.093372 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efabfa7a-c497-4ae1-a44c-cfb86515973d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:54.095013 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efabfa7a-c497-4ae1-a44c-cfb86515973d admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:54.110262 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1183/2366] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:25:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:25:54.112265 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfa99cf-d3cd-4c13-babe-96b2ee01f135 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:54.122784 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfa99cf-d3cd-4c13-babe-96b2ee01f135 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:54.128807 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b2a9cb-5338-4147-8127-c2b56752b589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:54.129790 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b2a9cb-5338-4147-8127-c2b56752b589 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:54.130191 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b2a9cb-5338-4147-8127-c2b56752b589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:54.130614 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b2a9cb-5338-4147-8127-c2b56752b589 admin admin] PATH_INFO: `/v3/projects/bc7dbcccb398405d9aacfee9c8a1e219/users/7f0bbdc4af4d491d9722b71fa5f1586c/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:54.150399 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b2a9cb-5338-4147-8127-c2b56752b589 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc7dbcccb398405d9aacfee9c8a1e219, user_id=7f0bbdc4af4d491d9722b71fa5f1586c, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:54.153121 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b2a9cb-5338-4147-8127-c2b56752b589 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:54.169572 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1184/2367] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:25:54 2022] PUT /identity/v3/projects/bc7dbcccb398405d9aacfee9c8a1e219/users/7f0bbdc4af4d491d9722b71fa5f1586c/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:25:54.180773 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c8e37f-459d-4939-8c01-dde1b23cc601 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:54.181099 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c8e37f-459d-4939-8c01-dde1b23cc601 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:54.181393 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c8e37f-459d-4939-8c01-dde1b23cc601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:54.211715 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-86c8e37f-459d-4939-8c01-dde1b23cc601 None None] MFA Rules not processed for user `7f0bbdc4af4d491d9722b71fa5f1586c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:25:54.213012 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-86c8e37f-459d-4939-8c01-dde1b23cc601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:25:54.292431 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1184/2368] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:25:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:25:54.295801 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b2a9cb-5338-4147-8127-c2b56752b589 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:54.327565 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:25:54.373963 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b2a9cb-5338-4147-8127-c2b56752b589 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:54.379363 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20bad1f-41df-4468-9a14-9c6aea915293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:54.380035 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20bad1f-41df-4468-9a14-9c6aea915293 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:54.380334 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20bad1f-41df-4468-9a14-9c6aea915293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:54.380658 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20bad1f-41df-4468-9a14-9c6aea915293 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:54.381440 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20bad1f-41df-4468-9a14-9c6aea915293 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:54.383346 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20bad1f-41df-4468-9a14-9c6aea915293 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:54.413997 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1185/2369] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:25:54 2022] POST /identity/v3/projects => generated 375 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:25:54.415187 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c8e37f-459d-4939-8c01-dde1b23cc601 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:54.432070 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c8e37f-459d-4939-8c01-dde1b23cc601 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:54.453650 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:25:54.520594 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe067930-5dcf-45c5-ba0d-d69cd3830079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:54.521401 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe067930-5dcf-45c5-ba0d-d69cd3830079 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:54.521817 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe067930-5dcf-45c5-ba0d-d69cd3830079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:54.522164 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe067930-5dcf-45c5-ba0d-d69cd3830079 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:54.522945 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe067930-5dcf-45c5-ba0d-d69cd3830079 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:54.525492 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe067930-5dcf-45c5-ba0d-d69cd3830079 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:54.552445 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1185/2370] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:25:54 2022] POST /identity/v3/users => generated 294 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:25:54.553677 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20bad1f-41df-4468-9a14-9c6aea915293 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:54.564522 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20bad1f-41df-4468-9a14-9c6aea915293 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:54.570332 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3318163e-a943-4582-96cd-c076eb93d3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:54.571093 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3318163e-a943-4582-96cd-c076eb93d3e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:54.571442 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3318163e-a943-4582-96cd-c076eb93d3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:54.571789 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3318163e-a943-4582-96cd-c076eb93d3e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:54.572592 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3318163e-a943-4582-96cd-c076eb93d3e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:54.574372 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3318163e-a943-4582-96cd-c076eb93d3e0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:54.588782 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1186/2371] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:25:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:25:54.590157 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe067930-5dcf-45c5-ba0d-d69cd3830079 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:54.600682 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe067930-5dcf-45c5-ba0d-d69cd3830079 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:54.605360 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c88761c-17b9-46bf-a38d-f5b5a749e52a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:54.606240 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c88761c-17b9-46bf-a38d-f5b5a749e52a admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:54.606518 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c88761c-17b9-46bf-a38d-f5b5a749e52a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:54.606793 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c88761c-17b9-46bf-a38d-f5b5a749e52a admin admin] PATH_INFO: `/v3/projects/2a58079f42e946608627ec1a3e7251a3/users/98b01eba883d4352bae4e5588cfd4de1/roles/f36e053b442540f08041ac7878760f65` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:54.622285 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c88761c-17b9-46bf-a38d-f5b5a749e52a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a58079f42e946608627ec1a3e7251a3, user_id=98b01eba883d4352bae4e5588cfd4de1, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:54.624947 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c88761c-17b9-46bf-a38d-f5b5a749e52a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:54.639479 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1186/2372] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:25:54 2022] PUT /identity/v3/projects/2a58079f42e946608627ec1a3e7251a3/users/98b01eba883d4352bae4e5588cfd4de1/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:25:54.645028 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3318163e-a943-4582-96cd-c076eb93d3e0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:54.657443 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3318163e-a943-4582-96cd-c076eb93d3e0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:54.724668 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317d954f-787b-468e-9bac-8243b83f0f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:54.725771 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d954f-787b-468e-9bac-8243b83f0f01 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:54.726054 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d954f-787b-468e-9bac-8243b83f0f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:54.726333 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d954f-787b-468e-9bac-8243b83f0f01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:54.727055 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317d954f-787b-468e-9bac-8243b83f0f01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:54.728639 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317d954f-787b-468e-9bac-8243b83f0f01 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:54.739984 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1187/2373] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:25:54 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:25:54.744209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c88761c-17b9-46bf-a38d-f5b5a749e52a admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:54.756233 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c88761c-17b9-46bf-a38d-f5b5a749e52a admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:54.761003 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c779b4-7ab3-4d14-bacb-d16febc8ae24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:54.761924 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c779b4-7ab3-4d14-bacb-d16febc8ae24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:54.762208 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c779b4-7ab3-4d14-bacb-d16febc8ae24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:54.762486 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c779b4-7ab3-4d14-bacb-d16febc8ae24 admin admin] PATH_INFO: `/v3/projects/2a58079f42e946608627ec1a3e7251a3/users/98b01eba883d4352bae4e5588cfd4de1/roles/f36e053b442540f08041ac7878760f65` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:54.764425 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c779b4-7ab3-4d14-bacb-d16febc8ae24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a58079f42e946608627ec1a3e7251a3, user_id=98b01eba883d4352bae4e5588cfd4de1, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:54.767396 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c779b4-7ab3-4d14-bacb-d16febc8ae24 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:54.778185 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1187/2374] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:25:54 2022] PUT /identity/v3/projects/2a58079f42e946608627ec1a3e7251a3/users/98b01eba883d4352bae4e5588cfd4de1/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:25:54.788460 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaa0628-eec7-475d-ac60-e71957e2b296 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:54.788761 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaa0628-eec7-475d-ac60-e71957e2b296 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:54.789057 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaa0628-eec7-475d-ac60-e71957e2b296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:54.817745 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3eaa0628-eec7-475d-ac60-e71957e2b296 None None] MFA Rules not processed for user `98b01eba883d4352bae4e5588cfd4de1`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:25:54.819435 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3eaa0628-eec7-475d-ac60-e71957e2b296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:25:54.912367 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1188/2375] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:25:54 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:25:54.923108 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c779b4-7ab3-4d14-bacb-d16febc8ae24 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:54.925666 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d0c779b4-7ab3-4d14-bacb-d16febc8ae24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:25:54.983643 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c779b4-7ab3-4d14-bacb-d16febc8ae24 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:54.989203 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab02df04-e426-4006-81d7-924649517c33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:54.989853 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab02df04-e426-4006-81d7-924649517c33 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:54.990136 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab02df04-e426-4006-81d7-924649517c33 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:54.990410 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab02df04-e426-4006-81d7-924649517c33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:55.054428 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab02df04-e426-4006-81d7-924649517c33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:55.055894 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab02df04-e426-4006-81d7-924649517c33 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:55.065950 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1188/2376] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:25:54 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:26:00.435254 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaa0628-eec7-475d-ac60-e71957e2b296 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:00.438080 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3eaa0628-eec7-475d-ac60-e71957e2b296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:00.495722 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:00.595455 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaa0628-eec7-475d-ac60-e71957e2b296 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:00.605005 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7ee985-55e7-4e4f-a17b-fbc0ae16b01f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15039da209ec445d93d49e2bd6d854eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:00.605802 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ee985-55e7-4e4f-a17b-fbc0ae16b01f service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:00.606189 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ee985-55e7-4e4f-a17b-fbc0ae16b01f service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:00.606569 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ee985-55e7-4e4f-a17b-fbc0ae16b01f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:00.609076 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3d7ee985-55e7-4e4f-a17b-fbc0ae16b01f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:00.634355 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7ee985-55e7-4e4f-a17b-fbc0ae16b01f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:00.636253 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7ee985-55e7-4e4f-a17b-fbc0ae16b01f service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:00.651295 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1189/2377] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:26:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:26:00.823601 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab02df04-e426-4006-81d7-924649517c33 service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:00.834219 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab02df04-e426-4006-81d7-924649517c33 service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:00.841325 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b76f77-ed2e-4443-bb3f-1915286ba598 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:00.842513 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b76f77-ed2e-4443-bb3f-1915286ba598 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:00.842958 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b76f77-ed2e-4443-bb3f-1915286ba598 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:00.843637 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b76f77-ed2e-4443-bb3f-1915286ba598 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:00.863879 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b76f77-ed2e-4443-bb3f-1915286ba598 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:00.866372 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b76f77-ed2e-4443-bb3f-1915286ba598 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:00.887554 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1189/2378] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:26:00 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:26:01.183431 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7ee985-55e7-4e4f-a17b-fbc0ae16b01f service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:01.185063 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3d7ee985-55e7-4e4f-a17b-fbc0ae16b01f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:01.263072 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7ee985-55e7-4e4f-a17b-fbc0ae16b01f service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:01.267822 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16d71d0-4405-4d5d-b190-0daf06c4db70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:01.268451 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d71d0-4405-4d5d-b190-0daf06c4db70 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:01.268740 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d71d0-4405-4d5d-b190-0daf06c4db70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:01.269021 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d71d0-4405-4d5d-b190-0daf06c4db70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:01.271388 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f16d71d0-4405-4d5d-b190-0daf06c4db70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:01.387807 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16d71d0-4405-4d5d-b190-0daf06c4db70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:01.393094 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16d71d0-4405-4d5d-b190-0daf06c4db70 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:01.404969 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1190/2379] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:26:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:26:02.109676 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b76f77-ed2e-4443-bb3f-1915286ba598 service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:02.111187 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f7b76f77-ed2e-4443-bb3f-1915286ba598 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:02.134412 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b76f77-ed2e-4443-bb3f-1915286ba598 service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:02.147917 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7f45f6-ac19-4cd0-bf32-c78d42903a52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:02.148654 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f45f6-ac19-4cd0-bf32-c78d42903a52 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:02.148946 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f45f6-ac19-4cd0-bf32-c78d42903a52 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:02.149255 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f45f6-ac19-4cd0-bf32-c78d42903a52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:02.151133 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-9b7f45f6-ac19-4cd0-bf32-c78d42903a52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:02.176285 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7f45f6-ac19-4cd0-bf32-c78d42903a52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:02.177718 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7f45f6-ac19-4cd0-bf32-c78d42903a52 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:02.197023 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1190/2380] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:26:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:26:05.368947 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16d71d0-4405-4d5d-b190-0daf06c4db70 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:05.370446 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f16d71d0-4405-4d5d-b190-0daf06c4db70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:05.397862 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16d71d0-4405-4d5d-b190-0daf06c4db70 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:05.407129 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5c8558-e963-48dc-9d3c-b71d7440d753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:05.411637 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c8558-e963-48dc-9d3c-b71d7440d753 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:05.411637 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c8558-e963-48dc-9d3c-b71d7440d753 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:05.411637 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c8558-e963-48dc-9d3c-b71d7440d753 admin admin] PATH_INFO: `/v3/users/e50f9b53a10a43fa9b5fec03d01f451f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:05.411637 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5c8558-e963-48dc-9d3c-b71d7440d753 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e50f9b53a10a43fa9b5fec03d01f451f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:05.412597 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5c8558-e963-48dc-9d3c-b71d7440d753 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:05.497544 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3f5c8558-e963-48dc-9d3c-b71d7440d753 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e50f9b53a10a43fa9b5fec03d01f451f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:05.498471 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3f5c8558-e963-48dc-9d3c-b71d7440d753 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e50f9b53a10a43fa9b5fec03d01f451f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:05.502808 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:05.512700 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3f5c8558-e963-48dc-9d3c-b71d7440d753 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e50f9b53a10a43fa9b5fec03d01f451f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:05.546320 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3f5c8558-e963-48dc-9d3c-b71d7440d753 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e50f9b53a10a43fa9b5fec03d01f451f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:05.547290 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3f5c8558-e963-48dc-9d3c-b71d7440d753 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e50f9b53a10a43fa9b5fec03d01f451f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:05.567775 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1191/2381] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:26:05 2022] DELETE /identity/v3/users/e50f9b53a10a43fa9b5fec03d01f451f => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:05.877949 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7f45f6-ac19-4cd0-bf32-c78d42903a52 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:05.885681 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:05.894783 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7f45f6-ac19-4cd0-bf32-c78d42903a52 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:05.978248 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363b71a6-32f1-4afe-bb9a-3808b44060a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:05.979907 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363b71a6-32f1-4afe-bb9a-3808b44060a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:05.980318 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363b71a6-32f1-4afe-bb9a-3808b44060a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:05.980670 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363b71a6-32f1-4afe-bb9a-3808b44060a4 admin admin] PATH_INFO: `/v3/projects/123c769553a1424e9c6dba4569bfaea5` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:05.982640 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363b71a6-32f1-4afe-bb9a-3808b44060a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=123c769553a1424e9c6dba4569bfaea5)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:05.984845 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363b71a6-32f1-4afe-bb9a-3808b44060a4 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:06.043271 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-363b71a6-32f1-4afe-bb9a-3808b44060a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '123c769553a1424e9c6dba4569bfaea5', 'request_id': 'req-363b71a6-32f1-4afe-bb9a-3808b44060a4'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:06.049968 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-363b71a6-32f1-4afe-bb9a-3808b44060a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '123c769553a1424e9c6dba4569bfaea5', 'request_id': 'req-363b71a6-32f1-4afe-bb9a-3808b44060a4'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:06.057830 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-363b71a6-32f1-4afe-bb9a-3808b44060a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '123c769553a1424e9c6dba4569bfaea5', 'request_id': 'req-363b71a6-32f1-4afe-bb9a-3808b44060a4'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:06.073852 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-363b71a6-32f1-4afe-bb9a-3808b44060a4 admin admin] The token cache is being invalidate because project 123c769553a1424e9c6dba4569bfaea5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:26:06.074150 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-363b71a6-32f1-4afe-bb9a-3808b44060a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:06.074806 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-363b71a6-32f1-4afe-bb9a-3808b44060a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:06.080660 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1191/2382] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:26:05 2022] DELETE /identity/v3/projects/123c769553a1424e9c6dba4569bfaea5 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:06.081714 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd8168-adc8-4b8f-8309-d10e2fdf1c93 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:06.082091 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd8168-adc8-4b8f-8309-d10e2fdf1c93 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:06.082273 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd8168-adc8-4b8f-8309-d10e2fdf1c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:06.107044 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e7fd8168-adc8-4b8f-8309-d10e2fdf1c93 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:06.108471 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e7fd8168-adc8-4b8f-8309-d10e2fdf1c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:06.175498 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1192/2383] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:26:06.183228 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4002951-a160-4dbf-ae34-15a4aba933c6 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:06.183529 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4002951-a160-4dbf-ae34-15a4aba933c6 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:06.183819 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4002951-a160-4dbf-ae34-15a4aba933c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:06.214669 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-e4002951-a160-4dbf-ae34-15a4aba933c6 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:06.216028 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e4002951-a160-4dbf-ae34-15a4aba933c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:06.228617 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1192/2384] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:26:06.229787 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fd8168-adc8-4b8f-8309-d10e2fdf1c93 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:06.244419 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fd8168-adc8-4b8f-8309-d10e2fdf1c93 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:06.250448 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218c9342-9843-4b86-bf6e-3a9bbdd14c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:06.251274 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c9342-9843-4b86-bf6e-3a9bbdd14c27 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:06.251623 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c9342-9843-4b86-bf6e-3a9bbdd14c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:06.252009 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c9342-9843-4b86-bf6e-3a9bbdd14c27 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:06.252878 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218c9342-9843-4b86-bf6e-3a9bbdd14c27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:26:06.253298 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218c9342-9843-4b86-bf6e-3a9bbdd14c27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:06.255020 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218c9342-9843-4b86-bf6e-3a9bbdd14c27 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:06.272104 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1193/2385] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:26:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:26:06.273818 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4002951-a160-4dbf-ae34-15a4aba933c6 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:06.286222 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4002951-a160-4dbf-ae34-15a4aba933c6 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:06.292584 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3acbe2-33d5-4d13-b1af-72c8ad2a4edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:06.293398 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3acbe2-33d5-4d13-b1af-72c8ad2a4edf admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:06.293807 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3acbe2-33d5-4d13-b1af-72c8ad2a4edf admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:06.294156 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3acbe2-33d5-4d13-b1af-72c8ad2a4edf admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:06.295058 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3acbe2-33d5-4d13-b1af-72c8ad2a4edf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:06.297067 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3acbe2-33d5-4d13-b1af-72c8ad2a4edf admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:06.339005 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1193/2386] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:26:06 2022] POST /identity/v3/projects => generated 373 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:26:06.339805 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218c9342-9843-4b86-bf6e-3a9bbdd14c27 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:06.359202 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218c9342-9843-4b86-bf6e-3a9bbdd14c27 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:06.466593 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6f1604-c96a-426b-89c5-1ff9c0d70e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:06.467358 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f1604-c96a-426b-89c5-1ff9c0d70e29 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:06.467891 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f1604-c96a-426b-89c5-1ff9c0d70e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:06.468647 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f1604-c96a-426b-89c5-1ff9c0d70e29 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:06.470411 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6f1604-c96a-426b-89c5-1ff9c0d70e29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:06.474021 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6f1604-c96a-426b-89c5-1ff9c0d70e29 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:06.503911 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1194/2387] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:26:06 2022] POST /identity/v3/users => generated 287 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:26:06.507918 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3acbe2-33d5-4d13-b1af-72c8ad2a4edf admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:06.522212 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3acbe2-33d5-4d13-b1af-72c8ad2a4edf admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:06.533063 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e231762b-635e-4c1f-81dd-b6feac5bb572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:06.535011 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e231762b-635e-4c1f-81dd-b6feac5bb572 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:06.535394 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e231762b-635e-4c1f-81dd-b6feac5bb572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:06.535771 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e231762b-635e-4c1f-81dd-b6feac5bb572 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:06.536701 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e231762b-635e-4c1f-81dd-b6feac5bb572 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:06.547595 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e231762b-635e-4c1f-81dd-b6feac5bb572 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:06.571304 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1194/2388] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:26:06 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:26:06.572775 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6f1604-c96a-426b-89c5-1ff9c0d70e29 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:06.585799 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6f1604-c96a-426b-89c5-1ff9c0d70e29 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:06.594318 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4def523c-4033-42a4-8bff-941267fd9ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:06.595131 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4def523c-4033-42a4-8bff-941267fd9ce0 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:06.595491 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4def523c-4033-42a4-8bff-941267fd9ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:06.596441 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4def523c-4033-42a4-8bff-941267fd9ce0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:06.597328 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4def523c-4033-42a4-8bff-941267fd9ce0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:06.598989 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4def523c-4033-42a4-8bff-941267fd9ce0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:06.613862 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1195/2389] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:26:06 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:26:06.618030 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e231762b-635e-4c1f-81dd-b6feac5bb572 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:06.636124 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e231762b-635e-4c1f-81dd-b6feac5bb572 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:06.651127 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eafb3f-8a2a-436b-a641-d76a9543ba82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:06.652035 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eafb3f-8a2a-436b-a641-d76a9543ba82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:06.652324 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eafb3f-8a2a-436b-a641-d76a9543ba82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:06.652606 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eafb3f-8a2a-436b-a641-d76a9543ba82 admin admin] PATH_INFO: `/v3/projects/888e470954834c2796e4d1872fa14e5c/users/eee5f003c6274bd6b27e8de6f262491c/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:06.670039 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eafb3f-8a2a-436b-a641-d76a9543ba82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=888e470954834c2796e4d1872fa14e5c, user_id=eee5f003c6274bd6b27e8de6f262491c, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:06.672787 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eafb3f-8a2a-436b-a641-d76a9543ba82 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:06.706025 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1195/2390] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:26:06 2022] PUT /identity/v3/projects/888e470954834c2796e4d1872fa14e5c/users/eee5f003c6274bd6b27e8de6f262491c/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:06.716759 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893f870-c624-41c2-889f-65b42c95e5e4 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:06.716953 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893f870-c624-41c2-889f-65b42c95e5e4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:06.717263 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893f870-c624-41c2-889f-65b42c95e5e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:06.780642 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a893f870-c624-41c2-889f-65b42c95e5e4 None None] MFA Rules not processed for user `eee5f003c6274bd6b27e8de6f262491c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:06.785096 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a893f870-c624-41c2-889f-65b42c95e5e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:06.834937 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:06.915034 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1196/2391] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:26:06.916316 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eafb3f-8a2a-436b-a641-d76a9543ba82 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:06.972397 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:07.045162 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eafb3f-8a2a-436b-a641-d76a9543ba82 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:07.052874 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1153b08-78dc-42a1-a169-021a4f933392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:07.054099 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1153b08-78dc-42a1-a169-021a4f933392 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:07.054533 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1153b08-78dc-42a1-a169-021a4f933392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:07.054827 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1153b08-78dc-42a1-a169-021a4f933392 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:07.055640 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1153b08-78dc-42a1-a169-021a4f933392 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:07.058876 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1153b08-78dc-42a1-a169-021a4f933392 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:07.104940 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1196/2392] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:26:06 2022] POST /identity/v3/projects => generated 371 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:26:07.109965 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a893f870-c624-41c2-889f-65b42c95e5e4 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:07.125741 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a893f870-c624-41c2-889f-65b42c95e5e4 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:07.202675 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9788cc7-ceb8-46aa-b397-20ef078f294a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:07.203897 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9788cc7-ceb8-46aa-b397-20ef078f294a admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:07.206585 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9788cc7-ceb8-46aa-b397-20ef078f294a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:07.214847 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9788cc7-ceb8-46aa-b397-20ef078f294a admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:07.216662 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9788cc7-ceb8-46aa-b397-20ef078f294a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:07.219390 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9788cc7-ceb8-46aa-b397-20ef078f294a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:07.281951 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1197/2393] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:26:07 2022] POST /identity/v3/users => generated 292 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:26:07.286739 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1153b08-78dc-42a1-a169-021a4f933392 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:07.302837 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1153b08-78dc-42a1-a169-021a4f933392 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:07.318639 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac2a9c9-2cf8-461f-a2b3-6966eceab03d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:07.319234 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2a9c9-2cf8-461f-a2b3-6966eceab03d admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:07.319579 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2a9c9-2cf8-461f-a2b3-6966eceab03d admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:07.319862 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2a9c9-2cf8-461f-a2b3-6966eceab03d admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:07.320680 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac2a9c9-2cf8-461f-a2b3-6966eceab03d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:07.322557 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac2a9c9-2cf8-461f-a2b3-6966eceab03d admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:07.341330 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1197/2394] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:26:07.345580 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9788cc7-ceb8-46aa-b397-20ef078f294a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:07.363252 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9788cc7-ceb8-46aa-b397-20ef078f294a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:07.372414 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec38cf98-ac6d-45e7-88a7-b683e1b70a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:07.383557 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38cf98-ac6d-45e7-88a7-b683e1b70a89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:07.389530 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38cf98-ac6d-45e7-88a7-b683e1b70a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:07.389994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38cf98-ac6d-45e7-88a7-b683e1b70a89 admin admin] PATH_INFO: `/v3/projects/02b8a4dafba54c229d0650e915f774d2/users/09230860bc9e48c0a6bf9106551fa063/roles/f36e053b442540f08041ac7878760f65` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:07.421633 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec38cf98-ac6d-45e7-88a7-b683e1b70a89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02b8a4dafba54c229d0650e915f774d2, user_id=09230860bc9e48c0a6bf9106551fa063, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:07.426319 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec38cf98-ac6d-45e7-88a7-b683e1b70a89 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:07.451077 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1198/2395] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:26:07 2022] PUT /identity/v3/projects/02b8a4dafba54c229d0650e915f774d2/users/09230860bc9e48c0a6bf9106551fa063/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:07.454702 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac2a9c9-2cf8-461f-a2b3-6966eceab03d admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:07.467132 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac2a9c9-2cf8-461f-a2b3-6966eceab03d admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:07.514084 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec38cf98-ac6d-45e7-88a7-b683e1b70a89 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:07.516132 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ec38cf98-ac6d-45e7-88a7-b683e1b70a89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:07.576202 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f6a9d8-c033-47eb-abeb-aa5d64333233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:07.582696 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f6a9d8-c033-47eb-abeb-aa5d64333233 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:07.584002 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f6a9d8-c033-47eb-abeb-aa5d64333233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:07.585088 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f6a9d8-c033-47eb-abeb-aa5d64333233 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:07.586206 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f6a9d8-c033-47eb-abeb-aa5d64333233 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:07.589899 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f6a9d8-c033-47eb-abeb-aa5d64333233 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:07.603943 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1198/2396] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:26:07.607707 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f6a9d8-c033-47eb-abeb-aa5d64333233 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:07.623240 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f6a9d8-c033-47eb-abeb-aa5d64333233 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:07.651537 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81300302-d867-401d-b79f-d2f845390229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:07.655522 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81300302-d867-401d-b79f-d2f845390229 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:07.655928 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81300302-d867-401d-b79f-d2f845390229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:07.656322 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81300302-d867-401d-b79f-d2f845390229 admin admin] PATH_INFO: `/v3/projects/02b8a4dafba54c229d0650e915f774d2/users/09230860bc9e48c0a6bf9106551fa063/roles/f36e053b442540f08041ac7878760f65` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:07.661630 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81300302-d867-401d-b79f-d2f845390229 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02b8a4dafba54c229d0650e915f774d2, user_id=09230860bc9e48c0a6bf9106551fa063, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:07.668286 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec38cf98-ac6d-45e7-88a7-b683e1b70a89 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:07.668851 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81300302-d867-401d-b79f-d2f845390229 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:07.678186 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9855250-e98b-4fa4-b8b4-71072125c354 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '293c695f15c8437d81a619f840318f44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:07.678785 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9855250-e98b-4fa4-b8b4-71072125c354 service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:07.679124 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9855250-e98b-4fa4-b8b4-71072125c354 service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:07.679588 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9855250-e98b-4fa4-b8b4-71072125c354 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:07.681456 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a9855250-e98b-4fa4-b8b4-71072125c354 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:07.696150 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9855250-e98b-4fa4-b8b4-71072125c354 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:07.697813 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9855250-e98b-4fa4-b8b4-71072125c354 service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:07.698736 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1199/2397] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:26:07 2022] PUT /identity/v3/projects/02b8a4dafba54c229d0650e915f774d2/users/09230860bc9e48c0a6bf9106551fa063/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:26:07.699778 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81300302-d867-401d-b79f-d2f845390229 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:07.703021 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-81300302-d867-401d-b79f-d2f845390229 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:07.722466 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1199/2398] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:26:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:26:07.723904 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bd1f8d-436c-4617-8a9f-b062523d376a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:07.724191 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bd1f8d-436c-4617-8a9f-b062523d376a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:07.724469 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bd1f8d-436c-4617-8a9f-b062523d376a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:07.778495 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-16bd1f8d-436c-4617-8a9f-b062523d376a None None] MFA Rules not processed for user `09230860bc9e48c0a6bf9106551fa063`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:07.782278 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-16bd1f8d-436c-4617-8a9f-b062523d376a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:07.797234 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:07.847851 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81300302-d867-401d-b79f-d2f845390229 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:07.858928 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de42c41b-14e0-44b1-b506-31ef81584e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:07.860044 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42c41b-14e0-44b1-b506-31ef81584e48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:07.860441 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42c41b-14e0-44b1-b506-31ef81584e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:07.861082 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42c41b-14e0-44b1-b506-31ef81584e48 admin admin] PATH_INFO: `/v3/users/6bc30b7d536444008e5b5bab45c5c438` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:07.863404 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de42c41b-14e0-44b1-b506-31ef81584e48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bc30b7d536444008e5b5bab45c5c438)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:07.866285 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de42c41b-14e0-44b1-b506-31ef81584e48 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:07.905795 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:07.911859 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1200/2399] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:07 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:26:07.925706 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bd1f8d-436c-4617-8a9f-b062523d376a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:07.927391 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-16bd1f8d-436c-4617-8a9f-b062523d376a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:07.931878 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-de42c41b-14e0-44b1-b506-31ef81584e48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bc30b7d536444008e5b5bab45c5c438'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:07.941319 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-de42c41b-14e0-44b1-b506-31ef81584e48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bc30b7d536444008e5b5bab45c5c438'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:07.942191 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-de42c41b-14e0-44b1-b506-31ef81584e48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bc30b7d536444008e5b5bab45c5c438'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:07.952200 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-de42c41b-14e0-44b1-b506-31ef81584e48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bc30b7d536444008e5b5bab45c5c438'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:07.968614 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-de42c41b-14e0-44b1-b506-31ef81584e48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bc30b7d536444008e5b5bab45c5c438'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:07.979822 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1200/2400] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:26:07 2022] DELETE /identity/v3/users/6bc30b7d536444008e5b5bab45c5c438 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:07.993140 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bd1f8d-436c-4617-8a9f-b062523d376a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:08.063028 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812be28b-7379-4245-a7d1-d5b353d53bb8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:08.063695 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812be28b-7379-4245-a7d1-d5b353d53bb8 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:08.063984 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812be28b-7379-4245-a7d1-d5b353d53bb8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:08.064264 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812be28b-7379-4245-a7d1-d5b353d53bb8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:08.066822 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-812be28b-7379-4245-a7d1-d5b353d53bb8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:08.146429 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812be28b-7379-4245-a7d1-d5b353d53bb8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:08.147896 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812be28b-7379-4245-a7d1-d5b353d53bb8 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:08.158447 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1201/2401] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:26:07 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:26:08.196250 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de42c41b-14e0-44b1-b506-31ef81584e48 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:08.198022 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-de42c41b-14e0-44b1-b506-31ef81584e48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:08.254680 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812be28b-7379-4245-a7d1-d5b353d53bb8 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:08.256344 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-812be28b-7379-4245-a7d1-d5b353d53bb8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:08.270172 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812be28b-7379-4245-a7d1-d5b353d53bb8 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:08.274163 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed09c1d-39db-4d4c-89fa-5a05735b5f59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:08.275018 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed09c1d-39db-4d4c-89fa-5a05735b5f59 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:08.275611 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed09c1d-39db-4d4c-89fa-5a05735b5f59 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:08.276197 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed09c1d-39db-4d4c-89fa-5a05735b5f59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:08.278302 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2ed09c1d-39db-4d4c-89fa-5a05735b5f59 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:08.286701 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:08.313954 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de42c41b-14e0-44b1-b506-31ef81584e48 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:08.325126 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae43da-fe9f-4710-ad46-a5a6c86b9501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:08.326505 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae43da-fe9f-4710-ad46-a5a6c86b9501 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:08.326505 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae43da-fe9f-4710-ad46-a5a6c86b9501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:08.326832 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae43da-fe9f-4710-ad46-a5a6c86b9501 admin admin] PATH_INFO: `/v3/projects/ba157b0e49f1447e82755035db8728d5` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:08.328702 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ae43da-fe9f-4710-ad46-a5a6c86b9501 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba157b0e49f1447e82755035db8728d5)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:08.331579 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ae43da-fe9f-4710-ad46-a5a6c86b9501 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:08.364472 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed09c1d-39db-4d4c-89fa-5a05735b5f59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:08.366038 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed09c1d-39db-4d4c-89fa-5a05735b5f59 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:08.376492 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1202/2402] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:26:08 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:26:08.418229 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-72ae43da-fe9f-4710-ad46-a5a6c86b9501 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba157b0e49f1447e82755035db8728d5', 'request_id': 'req-72ae43da-fe9f-4710-ad46-a5a6c86b9501'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:08.430829 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-72ae43da-fe9f-4710-ad46-a5a6c86b9501 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba157b0e49f1447e82755035db8728d5', 'request_id': 'req-72ae43da-fe9f-4710-ad46-a5a6c86b9501'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:08.437947 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-72ae43da-fe9f-4710-ad46-a5a6c86b9501 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba157b0e49f1447e82755035db8728d5', 'request_id': 'req-72ae43da-fe9f-4710-ad46-a5a6c86b9501'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:08.474245 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-72ae43da-fe9f-4710-ad46-a5a6c86b9501 admin admin] The token cache is being invalidate because project ba157b0e49f1447e82755035db8728d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:26:08.474549 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-72ae43da-fe9f-4710-ad46-a5a6c86b9501 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:08.476135 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-72ae43da-fe9f-4710-ad46-a5a6c86b9501 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:08.478412 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1201/2403] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:26:08 2022] DELETE /identity/v3/projects/ba157b0e49f1447e82755035db8728d5 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:08.483784 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1c1d80-9d52-4602-8b41-20db45e051db None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:08.484382 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1c1d80-9d52-4602-8b41-20db45e051db None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:08.485196 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1c1d80-9d52-4602-8b41-20db45e051db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:08.508833 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-aa1c1d80-9d52-4602-8b41-20db45e051db None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:08.510336 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-aa1c1d80-9d52-4602-8b41-20db45e051db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:08.598584 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1203/2404] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:26:08.601112 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d4ccc-b168-4633-aa4c-620ae9c5cb49 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:08.601677 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d4ccc-b168-4633-aa4c-620ae9c5cb49 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:08.602588 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d4ccc-b168-4633-aa4c-620ae9c5cb49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:08.623710 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-e45d4ccc-b168-4633-aa4c-620ae9c5cb49 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:08.624930 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e45d4ccc-b168-4633-aa4c-620ae9c5cb49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:08.634425 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1202/2405] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:26:08.638758 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1c1d80-9d52-4602-8b41-20db45e051db None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:08.649789 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1c1d80-9d52-4602-8b41-20db45e051db None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:08.654618 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca17f92-c78e-40ab-a583-4251f71f809a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:08.655302 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca17f92-c78e-40ab-a583-4251f71f809a admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:08.655589 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca17f92-c78e-40ab-a583-4251f71f809a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:08.655874 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca17f92-c78e-40ab-a583-4251f71f809a admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:08.656668 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca17f92-c78e-40ab-a583-4251f71f809a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:26:08.656978 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca17f92-c78e-40ab-a583-4251f71f809a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:08.658769 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca17f92-c78e-40ab-a583-4251f71f809a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:08.671416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1204/2406] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:26:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:26:08.676085 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45d4ccc-b168-4633-aa4c-620ae9c5cb49 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:08.685846 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45d4ccc-b168-4633-aa4c-620ae9c5cb49 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:08.691375 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc80d8-2ce1-4324-baf0-e8e25bcbeb66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:08.692072 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc80d8-2ce1-4324-baf0-e8e25bcbeb66 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:08.692314 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc80d8-2ce1-4324-baf0-e8e25bcbeb66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:08.692594 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc80d8-2ce1-4324-baf0-e8e25bcbeb66 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:08.693342 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bc80d8-2ce1-4324-baf0-e8e25bcbeb66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:08.695321 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bc80d8-2ce1-4324-baf0-e8e25bcbeb66 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:08.722126 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1203/2407] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:26:08 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:26:08.726310 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca17f92-c78e-40ab-a583-4251f71f809a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:08.738170 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca17f92-c78e-40ab-a583-4251f71f809a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:08.799922 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d33a1e-4e4a-4474-b2fd-dbfcd106d12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:08.800613 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d33a1e-4e4a-4474-b2fd-dbfcd106d12e admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:08.800890 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d33a1e-4e4a-4474-b2fd-dbfcd106d12e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:08.801213 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d33a1e-4e4a-4474-b2fd-dbfcd106d12e admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:08.801921 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d33a1e-4e4a-4474-b2fd-dbfcd106d12e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:08.803875 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d33a1e-4e4a-4474-b2fd-dbfcd106d12e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:08.826579 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1205/2408] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:26:08 2022] POST /identity/v3/users => generated 283 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:26:08.830212 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc80d8-2ce1-4324-baf0-e8e25bcbeb66 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:08.840714 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc80d8-2ce1-4324-baf0-e8e25bcbeb66 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:08.847633 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189a951d-50fe-4ea8-9065-21b2ab87e887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:08.848099 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a951d-50fe-4ea8-9065-21b2ab87e887 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:08.848393 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a951d-50fe-4ea8-9065-21b2ab87e887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:08.848679 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a951d-50fe-4ea8-9065-21b2ab87e887 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:08.849559 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189a951d-50fe-4ea8-9065-21b2ab87e887 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:08.851212 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189a951d-50fe-4ea8-9065-21b2ab87e887 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:08.852697 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:08.869738 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1204/2409] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:26:08.870503 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d33a1e-4e4a-4474-b2fd-dbfcd106d12e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:08.882786 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d33a1e-4e4a-4474-b2fd-dbfcd106d12e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:08.886276 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48213df1-dc2e-4ae3-80d0-2e6cb9fcdddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:08.886955 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48213df1-dc2e-4ae3-80d0-2e6cb9fcdddb admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:08.887246 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48213df1-dc2e-4ae3-80d0-2e6cb9fcdddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:08.887530 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48213df1-dc2e-4ae3-80d0-2e6cb9fcdddb admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:08.888264 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48213df1-dc2e-4ae3-80d0-2e6cb9fcdddb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:08.889886 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48213df1-dc2e-4ae3-80d0-2e6cb9fcdddb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:08.904904 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1206/2410] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:26:08.906428 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189a951d-50fe-4ea8-9065-21b2ab87e887 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:08.916954 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189a951d-50fe-4ea8-9065-21b2ab87e887 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:08.922695 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ef62b4-ff94-4167-bf01-359f4be170d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:08.923076 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef62b4-ff94-4167-bf01-359f4be170d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:08.923369 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef62b4-ff94-4167-bf01-359f4be170d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:08.923663 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef62b4-ff94-4167-bf01-359f4be170d1 admin admin] PATH_INFO: `/v3/projects/48a447683caf42af93dbfe575ca3d9f0/users/369187e1b47a40b1851dc309d66d7a6c/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:08.942255 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ef62b4-ff94-4167-bf01-359f4be170d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48a447683caf42af93dbfe575ca3d9f0, user_id=369187e1b47a40b1851dc309d66d7a6c, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:08.945172 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ef62b4-ff94-4167-bf01-359f4be170d1 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:08.965081 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1205/2411] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:26:08 2022] PUT /identity/v3/projects/48a447683caf42af93dbfe575ca3d9f0/users/369187e1b47a40b1851dc309d66d7a6c/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:26:08.971478 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025712be-bf83-47e8-b66c-6318dada5cc7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:08.971478 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025712be-bf83-47e8-b66c-6318dada5cc7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:08.971733 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025712be-bf83-47e8-b66c-6318dada5cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:08.997148 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:09.000577 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-025712be-bf83-47e8-b66c-6318dada5cc7 None None] MFA Rules not processed for user `369187e1b47a40b1851dc309d66d7a6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:09.003253 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-025712be-bf83-47e8-b66c-6318dada5cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:09.067594 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1207/2412] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:08 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:26:09.078109 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ef62b4-ff94-4167-bf01-359f4be170d1 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:09.079808 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-56ef62b4-ff94-4167-bf01-359f4be170d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:09.136118 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ef62b4-ff94-4167-bf01-359f4be170d1 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:09.139735 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a668a9ed-80e1-4980-a853-d29f58771707 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:09.140369 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668a9ed-80e1-4980-a853-d29f58771707 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:09.140651 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668a9ed-80e1-4980-a853-d29f58771707 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:09.140926 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668a9ed-80e1-4980-a853-d29f58771707 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:09.149270 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a668a9ed-80e1-4980-a853-d29f58771707 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:09.150688 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a668a9ed-80e1-4980-a853-d29f58771707 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:09.159844 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1206/2413] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:26:09 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:26:18.774221 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008c00d1-034e-4bbd-b3b4-a874a423e047 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:18.774711 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008c00d1-034e-4bbd-b3b4-a874a423e047 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:18.774711 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008c00d1-034e-4bbd-b3b4-a874a423e047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:18.793226 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-008c00d1-034e-4bbd-b3b4-a874a423e047 None None] MFA Rules not processed for user `78668b939c5847b1b73e9c28b5d8db7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:18.794444 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-008c00d1-034e-4bbd-b3b4-a874a423e047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:18.852634 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1208/2414] 10.222.0.71 () {58 vars in 967 bytes} [Mon Jul 4 09:26:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:26:18.877366 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a668a9ed-80e1-4980-a853-d29f58771707 service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:18.878829 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a668a9ed-80e1-4980-a853-d29f58771707 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:18.953555 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a668a9ed-80e1-4980-a853-d29f58771707 service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:18.958609 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb243518-fb0a-4ff4-9a36-cc62ab2560ef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88b41b55769a4c449716c3590ed58225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:18.959472 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb243518-fb0a-4ff4-9a36-cc62ab2560ef service swift] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:18.959674 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb243518-fb0a-4ff4-9a36-cc62ab2560ef service swift] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:18.959962 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb243518-fb0a-4ff4-9a36-cc62ab2560ef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:18.971307 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb243518-fb0a-4ff4-9a36-cc62ab2560ef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:18.972974 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb243518-fb0a-4ff4-9a36-cc62ab2560ef service swift] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:18.982554 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1207/2415] 10.222.0.71 () {60 vars in 1384 bytes} [Mon Jul 4 09:26:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 09:26:24.170983 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008c00d1-034e-4bbd-b3b4-a874a423e047 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:24.172521 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-008c00d1-034e-4bbd-b3b4-a874a423e047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:24.259241 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008c00d1-034e-4bbd-b3b4-a874a423e047 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:24.265231 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53329cf3-1b64-4e35-9426-8a4322f7ac2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:24.265958 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53329cf3-1b64-4e35-9426-8a4322f7ac2b service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:24.266251 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53329cf3-1b64-4e35-9426-8a4322f7ac2b service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:24.266540 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53329cf3-1b64-4e35-9426-8a4322f7ac2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:24.269110 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-53329cf3-1b64-4e35-9426-8a4322f7ac2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:24.393175 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53329cf3-1b64-4e35-9426-8a4322f7ac2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:24.393175 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53329cf3-1b64-4e35-9426-8a4322f7ac2b service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:24.402692 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:24.414829 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1209/2416] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:26:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:26:25.030770 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb243518-fb0a-4ff4-9a36-cc62ab2560ef service swift] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:25.043340 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb243518-fb0a-4ff4-9a36-cc62ab2560ef service swift] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:25.064916 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b72ae0-c5e7-4f5d-ac74-15b557e312ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:25.066107 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b72ae0-c5e7-4f5d-ac74-15b557e312ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:25.066476 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b72ae0-c5e7-4f5d-ac74-15b557e312ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:25.066837 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b72ae0-c5e7-4f5d-ac74-15b557e312ca admin admin] PATH_INFO: `/v3/users/369187e1b47a40b1851dc309d66d7a6c` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:25.068705 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b72ae0-c5e7-4f5d-ac74-15b557e312ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=369187e1b47a40b1851dc309d66d7a6c)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:25.071874 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b72ae0-c5e7-4f5d-ac74-15b557e312ca admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:25.163070 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-29b72ae0-c5e7-4f5d-ac74-15b557e312ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '369187e1b47a40b1851dc309d66d7a6c'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:25.172383 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-29b72ae0-c5e7-4f5d-ac74-15b557e312ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '369187e1b47a40b1851dc309d66d7a6c'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:25.173591 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-29b72ae0-c5e7-4f5d-ac74-15b557e312ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '369187e1b47a40b1851dc309d66d7a6c'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:25.181654 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:25.185979 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-29b72ae0-c5e7-4f5d-ac74-15b557e312ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '369187e1b47a40b1851dc309d66d7a6c'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:25.212437 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-29b72ae0-c5e7-4f5d-ac74-15b557e312ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '369187e1b47a40b1851dc309d66d7a6c'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:25.222832 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1208/2417] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:26:25 2022] DELETE /identity/v3/users/369187e1b47a40b1851dc309d66d7a6c => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:25.224796 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b40568-aa0a-4dee-9cfc-adb9bdadac80 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:25.225284 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b40568-aa0a-4dee-9cfc-adb9bdadac80 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:25.225284 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b40568-aa0a-4dee-9cfc-adb9bdadac80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:25.252496 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-38b40568-aa0a-4dee-9cfc-adb9bdadac80 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:25.254074 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-38b40568-aa0a-4dee-9cfc-adb9bdadac80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:25.420532 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1210/2418] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:26:25.445458 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b72ae0-c5e7-4f5d-ac74-15b557e312ca admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:25.447235 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-29b72ae0-c5e7-4f5d-ac74-15b557e312ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:25.589308 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b72ae0-c5e7-4f5d-ac74-15b557e312ca admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:25.593786 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807e7ff4-9026-4409-a18f-060c4566d90c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:25.594495 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e7ff4-9026-4409-a18f-060c4566d90c service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:25.594787 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e7ff4-9026-4409-a18f-060c4566d90c service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:25.595157 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e7ff4-9026-4409-a18f-060c4566d90c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:25.612209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807e7ff4-9026-4409-a18f-060c4566d90c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:25.615727 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807e7ff4-9026-4409-a18f-060c4566d90c service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:25.644725 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1209/2419] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:26:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:26:26.449566 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b40568-aa0a-4dee-9cfc-adb9bdadac80 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:26.451176 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-38b40568-aa0a-4dee-9cfc-adb9bdadac80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:26.464771 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b40568-aa0a-4dee-9cfc-adb9bdadac80 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:26.474477 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6604ae9b-3629-449a-9895-7590950eefae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:26.475662 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6604ae9b-3629-449a-9895-7590950eefae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:26.476180 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6604ae9b-3629-449a-9895-7590950eefae admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:26.476868 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6604ae9b-3629-449a-9895-7590950eefae admin admin] PATH_INFO: `/v3/projects/48a447683caf42af93dbfe575ca3d9f0` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:26.478474 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6604ae9b-3629-449a-9895-7590950eefae admin admin] RBAC: Authorizing `identity:delete_project(project_id=48a447683caf42af93dbfe575ca3d9f0)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:26.481011 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6604ae9b-3629-449a-9895-7590950eefae admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:26.565388 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6604ae9b-3629-449a-9895-7590950eefae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48a447683caf42af93dbfe575ca3d9f0', 'request_id': 'req-6604ae9b-3629-449a-9895-7590950eefae'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:26.576353 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6604ae9b-3629-449a-9895-7590950eefae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48a447683caf42af93dbfe575ca3d9f0', 'request_id': 'req-6604ae9b-3629-449a-9895-7590950eefae'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:26.589783 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6604ae9b-3629-449a-9895-7590950eefae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48a447683caf42af93dbfe575ca3d9f0', 'request_id': 'req-6604ae9b-3629-449a-9895-7590950eefae'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:26.601606 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807e7ff4-9026-4409-a18f-060c4566d90c service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:26.603182 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-807e7ff4-9026-4409-a18f-060c4566d90c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:26.613579 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6604ae9b-3629-449a-9895-7590950eefae admin admin] The token cache is being invalidate because project 48a447683caf42af93dbfe575ca3d9f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:26:26.614181 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6604ae9b-3629-449a-9895-7590950eefae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:26.615039 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6604ae9b-3629-449a-9895-7590950eefae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:26.620548 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1211/2420] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:26:26 2022] DELETE /identity/v3/projects/48a447683caf42af93dbfe575ca3d9f0 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:26.622254 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25cb8a8-fae4-4799-bd96-d6afc7e9bace None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:26.622752 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25cb8a8-fae4-4799-bd96-d6afc7e9bace None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:26.623203 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25cb8a8-fae4-4799-bd96-d6afc7e9bace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:26.628533 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807e7ff4-9026-4409-a18f-060c4566d90c service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:26.648841 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d25cb8a8-fae4-4799-bd96-d6afc7e9bace None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:26.650480 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d25cb8a8-fae4-4799-bd96-d6afc7e9bace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:26.718967 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa03ac0-a431-403f-a7cf-ec7842ff9e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:26.720198 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa03ac0-a431-403f-a7cf-ec7842ff9e38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:26.720676 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa03ac0-a431-403f-a7cf-ec7842ff9e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:26.721252 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa03ac0-a431-403f-a7cf-ec7842ff9e38 admin admin] PATH_INFO: `/v3/users/b5ed81725b574b01aba6c604fbf9ed50` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:26.722687 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa03ac0-a431-403f-a7cf-ec7842ff9e38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5ed81725b574b01aba6c604fbf9ed50)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:26.724967 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa03ac0-a431-403f-a7cf-ec7842ff9e38 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:26.743600 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1212/2421] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:26:26.746043 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48f7bb-8d77-4c6e-93d7-c1d4c31403f0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:26.746423 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48f7bb-8d77-4c6e-93d7-c1d4c31403f0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:26.746787 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48f7bb-8d77-4c6e-93d7-c1d4c31403f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:26.773069 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-bf48f7bb-8d77-4c6e-93d7-c1d4c31403f0 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:26.777876 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bf48f7bb-8d77-4c6e-93d7-c1d4c31403f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:26.793779 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1213/2422] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:26:26.795257 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf48f7bb-8d77-4c6e-93d7-c1d4c31403f0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:26.811064 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf48f7bb-8d77-4c6e-93d7-c1d4c31403f0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:26.818990 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ca347-72c6-4c2c-8792-7240afece36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:26.820286 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ca347-72c6-4c2c-8792-7240afece36e admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:26.820716 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ca347-72c6-4c2c-8792-7240afece36e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:26.821165 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ca347-72c6-4c2c-8792-7240afece36e admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:26.822306 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960ca347-72c6-4c2c-8792-7240afece36e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:26:26.822933 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960ca347-72c6-4c2c-8792-7240afece36e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:26.824397 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960ca347-72c6-4c2c-8792-7240afece36e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:26.839370 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-5fa03ac0-a431-403f-a7cf-ec7842ff9e38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5ed81725b574b01aba6c604fbf9ed50'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:26.845855 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1214/2423] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:26:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:26:26.850644 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ca347-72c6-4c2c-8792-7240afece36e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:26.854248 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-5fa03ac0-a431-403f-a7cf-ec7842ff9e38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5ed81725b574b01aba6c604fbf9ed50'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:26.855865 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-5fa03ac0-a431-403f-a7cf-ec7842ff9e38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5ed81725b574b01aba6c604fbf9ed50'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:26.860177 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:26.867103 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ca347-72c6-4c2c-8792-7240afece36e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:26.874570 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-5fa03ac0-a431-403f-a7cf-ec7842ff9e38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5ed81725b574b01aba6c604fbf9ed50'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:26.878198 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b5d568-eb9d-4da0-bc85-787cebd712eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:26.879264 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b5d568-eb9d-4da0-bc85-787cebd712eb admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:26.880998 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b5d568-eb9d-4da0-bc85-787cebd712eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:26.881470 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b5d568-eb9d-4da0-bc85-787cebd712eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:26.883749 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b5d568-eb9d-4da0-bc85-787cebd712eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:26.887646 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b5d568-eb9d-4da0-bc85-787cebd712eb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:26.919572 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-5fa03ac0-a431-403f-a7cf-ec7842ff9e38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5ed81725b574b01aba6c604fbf9ed50'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:26.921630 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:26.930712 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1210/2424] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:26:26 2022] DELETE /identity/v3/users/b5ed81725b574b01aba6c604fbf9ed50 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:26.951599 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1215/2425] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:26:26 2022] POST /identity/v3/projects => generated 363 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:26:26.955547 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa03ac0-a431-403f-a7cf-ec7842ff9e38 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:26.957334 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-5fa03ac0-a431-403f-a7cf-ec7842ff9e38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:27.112220 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa03ac0-a431-403f-a7cf-ec7842ff9e38 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:27.208003 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b5d568-eb9d-4da0-bc85-787cebd712eb admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:27.210377 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-28b5d568-eb9d-4da0-bc85-787cebd712eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:27.250387 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1c9f33-6f16-4865-8fff-5331b867967f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:27.251504 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1c9f33-6f16-4865-8fff-5331b867967f admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:27.251955 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1c9f33-6f16-4865-8fff-5331b867967f admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:27.252245 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1c9f33-6f16-4865-8fff-5331b867967f admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:27.253517 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1c9f33-6f16-4865-8fff-5331b867967f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:27.256498 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1c9f33-6f16-4865-8fff-5331b867967f admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:27.290061 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1211/2426] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:26:26 2022] POST /identity/v3/users => generated 282 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:26:27.293997 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1c9f33-6f16-4865-8fff-5331b867967f admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:27.308768 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b5d568-eb9d-4da0-bc85-787cebd712eb admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:27.315739 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1c9f33-6f16-4865-8fff-5331b867967f admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:27.316150 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa17e116-06e9-49a7-8556-3c52e9ea4439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:27.317260 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa17e116-06e9-49a7-8556-3c52e9ea4439 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:27.317953 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa17e116-06e9-49a7-8556-3c52e9ea4439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:27.318321 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa17e116-06e9-49a7-8556-3c52e9ea4439 admin admin] PATH_INFO: `/v3/projects/c1411a14411442a1bded43af5f67decd` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:27.319568 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa17e116-06e9-49a7-8556-3c52e9ea4439 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1411a14411442a1bded43af5f67decd)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:27.321895 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa17e116-06e9-49a7-8556-3c52e9ea4439 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:27.330418 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8616c35b-2912-42ad-ac93-e6251a41490c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:27.331083 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616c35b-2912-42ad-ac93-e6251a41490c admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:27.331368 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616c35b-2912-42ad-ac93-e6251a41490c admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:27.331649 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616c35b-2912-42ad-ac93-e6251a41490c admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:27.332457 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8616c35b-2912-42ad-ac93-e6251a41490c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:27.334669 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8616c35b-2912-42ad-ac93-e6251a41490c admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:27.352605 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1212/2427] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:26:27 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:26:27.356850 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8616c35b-2912-42ad-ac93-e6251a41490c admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:27.399133 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8616c35b-2912-42ad-ac93-e6251a41490c admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:27.404235 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885141bd-3160-4d41-b225-23c6f90f4c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:27.404912 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885141bd-3160-4d41-b225-23c6f90f4c26 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:27.405229 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885141bd-3160-4d41-b225-23c6f90f4c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:27.406216 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885141bd-3160-4d41-b225-23c6f90f4c26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:27.406758 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885141bd-3160-4d41-b225-23c6f90f4c26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:27.409841 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885141bd-3160-4d41-b225-23c6f90f4c26 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:27.422992 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1213/2428] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:26:27 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:26:27.429933 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885141bd-3160-4d41-b225-23c6f90f4c26 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:27.444352 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-aa17e116-06e9-49a7-8556-3c52e9ea4439 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1411a14411442a1bded43af5f67decd', 'request_id': 'req-aa17e116-06e9-49a7-8556-3c52e9ea4439'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:27.453721 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885141bd-3160-4d41-b225-23c6f90f4c26 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:27.455677 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-aa17e116-06e9-49a7-8556-3c52e9ea4439 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1411a14411442a1bded43af5f67decd', 'request_id': 'req-aa17e116-06e9-49a7-8556-3c52e9ea4439'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:27.465084 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-aa17e116-06e9-49a7-8556-3c52e9ea4439 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1411a14411442a1bded43af5f67decd', 'request_id': 'req-aa17e116-06e9-49a7-8556-3c52e9ea4439'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:27.513087 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-aa17e116-06e9-49a7-8556-3c52e9ea4439 admin admin] The token cache is being invalidate because project c1411a14411442a1bded43af5f67decd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:26:27.513443 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-aa17e116-06e9-49a7-8556-3c52e9ea4439 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:27.514263 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-aa17e116-06e9-49a7-8556-3c52e9ea4439 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:27.518769 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1216/2429] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:26:27 2022] DELETE /identity/v3/projects/c1411a14411442a1bded43af5f67decd => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:27.523456 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa17e116-06e9-49a7-8556-3c52e9ea4439 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:27.526547 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-aa17e116-06e9-49a7-8556-3c52e9ea4439 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:27.567616 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d57ea7-aa11-46a5-b861-ecc6650bf1be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:27.569016 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d57ea7-aa11-46a5-b861-ecc6650bf1be admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:27.569356 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d57ea7-aa11-46a5-b861-ecc6650bf1be admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:27.569825 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d57ea7-aa11-46a5-b861-ecc6650bf1be admin admin] PATH_INFO: `/v3/projects/cf1fdcca16e84fe3907ea25eb02957ec/users/8b66d6b276964415b846d46ec5363782/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:27.588721 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d57ea7-aa11-46a5-b861-ecc6650bf1be admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf1fdcca16e84fe3907ea25eb02957ec, user_id=8b66d6b276964415b846d46ec5363782, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:27.592248 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d57ea7-aa11-46a5-b861-ecc6650bf1be admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:27.596350 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:27.611687 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1214/2430] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:26:27 2022] PUT /identity/v3/projects/cf1fdcca16e84fe3907ea25eb02957ec/users/8b66d6b276964415b846d46ec5363782/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:27.615719 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa17e116-06e9-49a7-8556-3c52e9ea4439 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:27.624415 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81922d-a36f-488d-a6c7-ff128b0fd8a5 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:27.624962 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81922d-a36f-488d-a6c7-ff128b0fd8a5 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:27.625440 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81922d-a36f-488d-a6c7-ff128b0fd8a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:27.651125 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:27.667645 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-ab81922d-a36f-488d-a6c7-ff128b0fd8a5 None None] MFA Rules not processed for user `8b66d6b276964415b846d46ec5363782`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:27.669823 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ab81922d-a36f-488d-a6c7-ff128b0fd8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:27.722969 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca4fc6a-db47-40f8-93b7-63bed9135387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:27.725511 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca4fc6a-db47-40f8-93b7-63bed9135387 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:27.726380 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca4fc6a-db47-40f8-93b7-63bed9135387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:27.727013 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca4fc6a-db47-40f8-93b7-63bed9135387 admin admin] PATH_INFO: `/v3/users/db53b4ed907444a19288bef6dec6a426` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:27.729257 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca4fc6a-db47-40f8-93b7-63bed9135387 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db53b4ed907444a19288bef6dec6a426)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:27.731752 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca4fc6a-db47-40f8-93b7-63bed9135387 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:27.767478 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1215/2431] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:27 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:26:27.773564 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab81922d-a36f-488d-a6c7-ff128b0fd8a5 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:27.775017 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ab81922d-a36f-488d-a6c7-ff128b0fd8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:27.792700 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab81922d-a36f-488d-a6c7-ff128b0fd8a5 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:27.797895 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af211ee3-ed29-47d9-8e11-67986aef235b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:27.798564 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af211ee3-ed29-47d9-8e11-67986aef235b admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:27.798856 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af211ee3-ed29-47d9-8e11-67986aef235b admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:27.799143 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af211ee3-ed29-47d9-8e11-67986aef235b admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:27.799846 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af211ee3-ed29-47d9-8e11-67986aef235b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:27.801823 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af211ee3-ed29-47d9-8e11-67986aef235b admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:27.806789 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2ca4fc6a-db47-40f8-93b7-63bed9135387 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db53b4ed907444a19288bef6dec6a426'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:27.807813 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2ca4fc6a-db47-40f8-93b7-63bed9135387 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db53b4ed907444a19288bef6dec6a426'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:27.818768 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2ca4fc6a-db47-40f8-93b7-63bed9135387 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db53b4ed907444a19288bef6dec6a426'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:27.841093 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1216/2432] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:26:27 2022] POST /identity/v3/projects => generated 365 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:26:27.842828 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af211ee3-ed29-47d9-8e11-67986aef235b admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:27.843028 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2ca4fc6a-db47-40f8-93b7-63bed9135387 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db53b4ed907444a19288bef6dec6a426'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:27.845446 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2ca4fc6a-db47-40f8-93b7-63bed9135387 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db53b4ed907444a19288bef6dec6a426'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:27.863575 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1217/2433] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:26:27 2022] DELETE /identity/v3/users/db53b4ed907444a19288bef6dec6a426 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:27.929864 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af211ee3-ed29-47d9-8e11-67986aef235b admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:27.992271 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b789fa-e418-42b3-9cc3-44ce49dec9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:27.992867 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b789fa-e418-42b3-9cc3-44ce49dec9b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:27.993186 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b789fa-e418-42b3-9cc3-44ce49dec9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:27.993470 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b789fa-e418-42b3-9cc3-44ce49dec9b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:27.994188 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b789fa-e418-42b3-9cc3-44ce49dec9b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:27.996092 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b789fa-e418-42b3-9cc3-44ce49dec9b3 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:28.012914 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:28.021634 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1217/2434] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:26:27 2022] POST /identity/v3/users => generated 289 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:26:28.026032 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca4fc6a-db47-40f8-93b7-63bed9135387 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:28.030446 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2ca4fc6a-db47-40f8-93b7-63bed9135387 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:28.031800 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b789fa-e418-42b3-9cc3-44ce49dec9b3 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:28.034281 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-09b789fa-e418-42b3-9cc3-44ce49dec9b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:28.044178 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca4fc6a-db47-40f8-93b7-63bed9135387 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:28.047571 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b789fa-e418-42b3-9cc3-44ce49dec9b3 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:28.050444 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ccae02-6783-4666-ae2b-42b5c9df1c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:28.051394 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ccae02-6783-4666-ae2b-42b5c9df1c8b admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:28.051918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ccae02-6783-4666-ae2b-42b5c9df1c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:28.052403 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e6b98b-5363-4311-bb74-31122efb0532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:28.053102 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ccae02-6783-4666-ae2b-42b5c9df1c8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:28.053304 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e6b98b-5363-4311-bb74-31122efb0532 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:28.053573 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e6b98b-5363-4311-bb74-31122efb0532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:28.053850 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e6b98b-5363-4311-bb74-31122efb0532 admin admin] PATH_INFO: `/v3/users/7f0bbdc4af4d491d9722b71fa5f1586c` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:28.055047 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ccae02-6783-4666-ae2b-42b5c9df1c8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:28.055177 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e6b98b-5363-4311-bb74-31122efb0532 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f0bbdc4af4d491d9722b71fa5f1586c)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:28.056998 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e6b98b-5363-4311-bb74-31122efb0532 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:28.057286 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ccae02-6783-4666-ae2b-42b5c9df1c8b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:28.071382 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1218/2435] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:26:28 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:26:28.075655 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ccae02-6783-4666-ae2b-42b5c9df1c8b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:28.088056 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ccae02-6783-4666-ae2b-42b5c9df1c8b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:28.094611 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14522b07-1f39-4c16-9b46-57eea1a71f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:28.101678 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14522b07-1f39-4c16-9b46-57eea1a71f25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:28.101975 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14522b07-1f39-4c16-9b46-57eea1a71f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:28.102917 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14522b07-1f39-4c16-9b46-57eea1a71f25 admin admin] PATH_INFO: `/v3/projects/8bef283e7e7e4e76b3c81e1e1cb6c060/users/756dff54b082436ca12786797a0b94d8/roles/f36e053b442540f08041ac7878760f65` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:28.123434 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14522b07-1f39-4c16-9b46-57eea1a71f25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bef283e7e7e4e76b3c81e1e1cb6c060, user_id=756dff54b082436ca12786797a0b94d8, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:28.126299 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14522b07-1f39-4c16-9b46-57eea1a71f25 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:28.146605 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1219/2436] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:26:28 2022] PUT /identity/v3/projects/8bef283e7e7e4e76b3c81e1e1cb6c060/users/756dff54b082436ca12786797a0b94d8/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:26:28.151067 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14522b07-1f39-4c16-9b46-57eea1a71f25 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:28.158270 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-39e6b98b-5363-4311-bb74-31122efb0532 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f0bbdc4af4d491d9722b71fa5f1586c'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:28.170482 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-39e6b98b-5363-4311-bb74-31122efb0532 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f0bbdc4af4d491d9722b71fa5f1586c'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:28.171304 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-39e6b98b-5363-4311-bb74-31122efb0532 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f0bbdc4af4d491d9722b71fa5f1586c'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:28.183273 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-39e6b98b-5363-4311-bb74-31122efb0532 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f0bbdc4af4d491d9722b71fa5f1586c'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:28.198798 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-39e6b98b-5363-4311-bb74-31122efb0532 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f0bbdc4af4d491d9722b71fa5f1586c'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:28.210359 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1218/2437] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:26:28 2022] DELETE /identity/v3/users/7f0bbdc4af4d491d9722b71fa5f1586c => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:28.215378 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e6b98b-5363-4311-bb74-31122efb0532 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:28.216956 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-39e6b98b-5363-4311-bb74-31122efb0532 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:28.282888 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14522b07-1f39-4c16-9b46-57eea1a71f25 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:28.289135 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:28.336220 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e6b98b-5363-4311-bb74-31122efb0532 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:28.350253 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1058cef6-a790-4054-bf1b-80e7bfee0f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:28.351145 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1058cef6-a790-4054-bf1b-80e7bfee0f83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:28.351434 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1058cef6-a790-4054-bf1b-80e7bfee0f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:28.351717 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1058cef6-a790-4054-bf1b-80e7bfee0f83 admin admin] PATH_INFO: `/v3/projects/bf83ec218a6846478ec086eda44cca45` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:28.352942 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1058cef6-a790-4054-bf1b-80e7bfee0f83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf83ec218a6846478ec086eda44cca45)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:28.354967 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1058cef6-a790-4054-bf1b-80e7bfee0f83 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:28.367707 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef4cbd9-fded-4b75-b9e9-3fda76bc0f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:28.368665 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef4cbd9-fded-4b75-b9e9-3fda76bc0f20 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:28.368966 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef4cbd9-fded-4b75-b9e9-3fda76bc0f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:28.369281 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef4cbd9-fded-4b75-b9e9-3fda76bc0f20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:28.370023 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef4cbd9-fded-4b75-b9e9-3fda76bc0f20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:28.371761 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef4cbd9-fded-4b75-b9e9-3fda76bc0f20 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:28.399545 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1220/2438] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:26:28 2022] GET /identity/v3/roles => generated 1420 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:26:28.403576 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1add8a-5a17-46d5-a456-24b2e4489e58 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:28.403899 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1add8a-5a17-46d5-a456-24b2e4489e58 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:28.404190 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1add8a-5a17-46d5-a456-24b2e4489e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:28.431199 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1f1add8a-5a17-46d5-a456-24b2e4489e58 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:28.432557 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1f1add8a-5a17-46d5-a456-24b2e4489e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:28.437985 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-1058cef6-a790-4054-bf1b-80e7bfee0f83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf83ec218a6846478ec086eda44cca45', 'request_id': 'req-1058cef6-a790-4054-bf1b-80e7bfee0f83'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:28.449991 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-1058cef6-a790-4054-bf1b-80e7bfee0f83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf83ec218a6846478ec086eda44cca45', 'request_id': 'req-1058cef6-a790-4054-bf1b-80e7bfee0f83'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:28.465960 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-1058cef6-a790-4054-bf1b-80e7bfee0f83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf83ec218a6846478ec086eda44cca45', 'request_id': 'req-1058cef6-a790-4054-bf1b-80e7bfee0f83'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:28.495019 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-1058cef6-a790-4054-bf1b-80e7bfee0f83 admin admin] The token cache is being invalidate because project bf83ec218a6846478ec086eda44cca45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:26:28.495356 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-1058cef6-a790-4054-bf1b-80e7bfee0f83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:28.496412 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-1058cef6-a790-4054-bf1b-80e7bfee0f83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:28.501499 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1219/2439] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:26:28 2022] DELETE /identity/v3/projects/bf83ec218a6846478ec086eda44cca45 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:28.503570 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1058cef6-a790-4054-bf1b-80e7bfee0f83 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:28.508394 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-1058cef6-a790-4054-bf1b-80e7bfee0f83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:28.523115 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:28.531896 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1221/2440] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:26:28.544537 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1add8a-5a17-46d5-a456-24b2e4489e58 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:28.550080 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1f1add8a-5a17-46d5-a456-24b2e4489e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:28.684381 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1058cef6-a790-4054-bf1b-80e7bfee0f83 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:28.694963 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1add8a-5a17-46d5-a456-24b2e4489e58 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:28.704656 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3796e4d9-b701-42e9-8b11-56b4e83cff92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:28.705431 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3796e4d9-b701-42e9-8b11-56b4e83cff92 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:28.705911 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3796e4d9-b701-42e9-8b11-56b4e83cff92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:28.706203 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3796e4d9-b701-42e9-8b11-56b4e83cff92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:28.706705 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5543e061-87cd-4663-8f52-da0af3d3eff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:28.709169 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5543e061-87cd-4663-8f52-da0af3d3eff6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:28.712344 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5543e061-87cd-4663-8f52-da0af3d3eff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:28.712932 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5543e061-87cd-4663-8f52-da0af3d3eff6 admin admin] PATH_INFO: `/v3/projects/8bef283e7e7e4e76b3c81e1e1cb6c060/users/756dff54b082436ca12786797a0b94d8/roles/f36e053b442540f08041ac7878760f65` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:28.718933 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5543e061-87cd-4663-8f52-da0af3d3eff6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bef283e7e7e4e76b3c81e1e1cb6c060, user_id=756dff54b082436ca12786797a0b94d8, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:28.722185 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5543e061-87cd-4663-8f52-da0af3d3eff6 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:28.734998 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3796e4d9-b701-42e9-8b11-56b4e83cff92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:28.736468 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3796e4d9-b701-42e9-8b11-56b4e83cff92 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:28.767788 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1220/2441] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:26:28 2022] PUT /identity/v3/projects/8bef283e7e7e4e76b3c81e1e1cb6c060/users/756dff54b082436ca12786797a0b94d8/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:28.774738 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110fc8f-4b28-40f9-8dfc-e52dcf84d7fb None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:28.775115 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1222/2442] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:26:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:26:28.775431 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110fc8f-4b28-40f9-8dfc-e52dcf84d7fb None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:28.775780 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110fc8f-4b28-40f9-8dfc-e52dcf84d7fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:28.845869 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-9110fc8f-4b28-40f9-8dfc-e52dcf84d7fb None None] MFA Rules not processed for user `756dff54b082436ca12786797a0b94d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:28.847335 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-9110fc8f-4b28-40f9-8dfc-e52dcf84d7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:28.959200 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1221/2443] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:28 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:26:28.972886 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3796e4d9-b701-42e9-8b11-56b4e83cff92 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:28.974984 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3796e4d9-b701-42e9-8b11-56b4e83cff92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:29.071006 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:29.085912 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3796e4d9-b701-42e9-8b11-56b4e83cff92 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:29.140825 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9110fc8f-4b28-40f9-8dfc-e52dcf84d7fb None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:29.142493 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-9110fc8f-4b28-40f9-8dfc-e52dcf84d7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:29.206739 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b0c5ba-0e76-4305-8c04-36658fa70cda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:29.207599 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b0c5ba-0e76-4305-8c04-36658fa70cda service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:29.207976 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b0c5ba-0e76-4305-8c04-36658fa70cda service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:29.209888 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b0c5ba-0e76-4305-8c04-36658fa70cda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:29.222809 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b0c5ba-0e76-4305-8c04-36658fa70cda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:29.225602 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b0c5ba-0e76-4305-8c04-36658fa70cda service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:29.237646 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1223/2444] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:26:28 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:26:29.240568 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b0c5ba-0e76-4305-8c04-36658fa70cda service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:29.242239 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-98b0c5ba-0e76-4305-8c04-36658fa70cda service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:29.250719 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9110fc8f-4b28-40f9-8dfc-e52dcf84d7fb None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:29.256631 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016ce562-d7f1-4b9a-ae5a-73c7cf67834e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:29.257485 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ce562-d7f1-4b9a-ae5a-73c7cf67834e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:29.257776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ce562-d7f1-4b9a-ae5a-73c7cf67834e admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:29.258060 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ce562-d7f1-4b9a-ae5a-73c7cf67834e admin admin] PATH_INFO: `/v3/users/60d06d081a29460b8e85b0b301399f08` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:29.259290 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016ce562-d7f1-4b9a-ae5a-73c7cf67834e admin admin] RBAC: Authorizing `identity:delete_user(user_id=60d06d081a29460b8e85b0b301399f08)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:29.259554 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b0c5ba-0e76-4305-8c04-36658fa70cda service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:29.261267 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016ce562-d7f1-4b9a-ae5a-73c7cf67834e admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:29.267173 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5d98dd-807e-4be4-ba3c-25c68d716150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:29.267507 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5d98dd-807e-4be4-ba3c-25c68d716150 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:29.267792 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5d98dd-807e-4be4-ba3c-25c68d716150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:29.268082 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5d98dd-807e-4be4-ba3c-25c68d716150 admin admin] PATH_INFO: `/v3/projects/bc7dbcccb398405d9aacfee9c8a1e219` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:29.269801 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5d98dd-807e-4be4-ba3c-25c68d716150 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc7dbcccb398405d9aacfee9c8a1e219)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:29.272598 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5d98dd-807e-4be4-ba3c-25c68d716150 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:29.292295 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:29.326957 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-016ce562-d7f1-4b9a-ae5a-73c7cf67834e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60d06d081a29460b8e85b0b301399f08'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:29.339950 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-016ce562-d7f1-4b9a-ae5a-73c7cf67834e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60d06d081a29460b8e85b0b301399f08'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:29.340698 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-016ce562-d7f1-4b9a-ae5a-73c7cf67834e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60d06d081a29460b8e85b0b301399f08'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:29.341788 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bf5d98dd-807e-4be4-ba3c-25c68d716150 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc7dbcccb398405d9aacfee9c8a1e219', 'request_id': 'req-bf5d98dd-807e-4be4-ba3c-25c68d716150'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:29.350676 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-016ce562-d7f1-4b9a-ae5a-73c7cf67834e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60d06d081a29460b8e85b0b301399f08'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:29.352813 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bf5d98dd-807e-4be4-ba3c-25c68d716150 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc7dbcccb398405d9aacfee9c8a1e219', 'request_id': 'req-bf5d98dd-807e-4be4-ba3c-25c68d716150'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:29.360844 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bf5d98dd-807e-4be4-ba3c-25c68d716150 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc7dbcccb398405d9aacfee9c8a1e219', 'request_id': 'req-bf5d98dd-807e-4be4-ba3c-25c68d716150'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:29.366660 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-016ce562-d7f1-4b9a-ae5a-73c7cf67834e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60d06d081a29460b8e85b0b301399f08'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:29.374285 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1222/2445] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:26:29 2022] DELETE /identity/v3/users/60d06d081a29460b8e85b0b301399f08 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:29.375308 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bf5d98dd-807e-4be4-ba3c-25c68d716150 admin admin] The token cache is being invalidate because project bc7dbcccb398405d9aacfee9c8a1e219 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:26:29.375465 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016ce562-d7f1-4b9a-ae5a-73c7cf67834e admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:29.375640 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bf5d98dd-807e-4be4-ba3c-25c68d716150 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:29.376250 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bf5d98dd-807e-4be4-ba3c-25c68d716150 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:29.377260 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-016ce562-d7f1-4b9a-ae5a-73c7cf67834e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:29.378191 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1224/2446] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:26:29 2022] DELETE /identity/v3/projects/bc7dbcccb398405d9aacfee9c8a1e219 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:29.382683 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5d98dd-807e-4be4-ba3c-25c68d716150 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:29.387722 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bf5d98dd-807e-4be4-ba3c-25c68d716150 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:29.477373 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016ce562-d7f1-4b9a-ae5a-73c7cf67834e admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:29.483639 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a25037f-7445-42ed-a36f-ebb336147478 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:29.484251 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a25037f-7445-42ed-a36f-ebb336147478 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:29.484807 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a25037f-7445-42ed-a36f-ebb336147478 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:29.485236 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a25037f-7445-42ed-a36f-ebb336147478 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:29.487959 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-5a25037f-7445-42ed-a36f-ebb336147478 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:29.504991 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5d98dd-807e-4be4-ba3c-25c68d716150 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:29.552617 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:29.580511 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a25037f-7445-42ed-a36f-ebb336147478 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:29.583959 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a25037f-7445-42ed-a36f-ebb336147478 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:29.586644 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8071d7d7-5f80-497c-9c9a-abbd6dc5a69d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:29.590014 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8071d7d7-5f80-497c-9c9a-abbd6dc5a69d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:29.590402 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8071d7d7-5f80-497c-9c9a-abbd6dc5a69d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:29.590768 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8071d7d7-5f80-497c-9c9a-abbd6dc5a69d admin admin] PATH_INFO: `/v3/users/98b01eba883d4352bae4e5588cfd4de1` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:29.595282 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8071d7d7-5f80-497c-9c9a-abbd6dc5a69d admin admin] RBAC: Authorizing `identity:delete_user(user_id=98b01eba883d4352bae4e5588cfd4de1)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:29.596646 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1223/2447] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:26:29 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:26:29.597876 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8071d7d7-5f80-497c-9c9a-abbd6dc5a69d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:29.627368 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a25037f-7445-42ed-a36f-ebb336147478 service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:29.629600 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-5a25037f-7445-42ed-a36f-ebb336147478 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:29.674099 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8071d7d7-5f80-497c-9c9a-abbd6dc5a69d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98b01eba883d4352bae4e5588cfd4de1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:29.674244 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a25037f-7445-42ed-a36f-ebb336147478 service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:29.675052 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8071d7d7-5f80-497c-9c9a-abbd6dc5a69d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98b01eba883d4352bae4e5588cfd4de1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:29.680982 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e279a1a0-6f90-4c88-9f0c-eb9954977c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:29.681894 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279a1a0-6f90-4c88-9f0c-eb9954977c19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:29.682634 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279a1a0-6f90-4c88-9f0c-eb9954977c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:29.682916 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279a1a0-6f90-4c88-9f0c-eb9954977c19 admin admin] PATH_INFO: `/v3/projects/83d33ddd4b124161aba270998b7d2462` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:29.684360 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e279a1a0-6f90-4c88-9f0c-eb9954977c19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83d33ddd4b124161aba270998b7d2462)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:29.686329 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e279a1a0-6f90-4c88-9f0c-eb9954977c19 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:29.691248 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8071d7d7-5f80-497c-9c9a-abbd6dc5a69d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98b01eba883d4352bae4e5588cfd4de1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:29.705438 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8071d7d7-5f80-497c-9c9a-abbd6dc5a69d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98b01eba883d4352bae4e5588cfd4de1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:29.706391 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8071d7d7-5f80-497c-9c9a-abbd6dc5a69d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98b01eba883d4352bae4e5588cfd4de1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:29.721546 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1225/2448] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:26:29 2022] DELETE /identity/v3/users/98b01eba883d4352bae4e5588cfd4de1 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:29.752260 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e279a1a0-6f90-4c88-9f0c-eb9954977c19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83d33ddd4b124161aba270998b7d2462', 'request_id': 'req-e279a1a0-6f90-4c88-9f0c-eb9954977c19'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:29.753514 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:29.762514 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e279a1a0-6f90-4c88-9f0c-eb9954977c19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83d33ddd4b124161aba270998b7d2462', 'request_id': 'req-e279a1a0-6f90-4c88-9f0c-eb9954977c19'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:29.771695 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e279a1a0-6f90-4c88-9f0c-eb9954977c19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83d33ddd4b124161aba270998b7d2462', 'request_id': 'req-e279a1a0-6f90-4c88-9f0c-eb9954977c19'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:29.794106 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e279a1a0-6f90-4c88-9f0c-eb9954977c19 admin admin] The token cache is being invalidate because project 83d33ddd4b124161aba270998b7d2462 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:26:29.794402 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e279a1a0-6f90-4c88-9f0c-eb9954977c19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:29.795159 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e279a1a0-6f90-4c88-9f0c-eb9954977c19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:29.797087 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1224/2449] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:26:29 2022] DELETE /identity/v3/projects/83d33ddd4b124161aba270998b7d2462 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:26:29.808461 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b4bb9a-b1d1-4151-b39d-624fe66757da None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:29.808823 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b4bb9a-b1d1-4151-b39d-624fe66757da None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:29.809075 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b4bb9a-b1d1-4151-b39d-624fe66757da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:29.828127 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-49b4bb9a-b1d1-4151-b39d-624fe66757da None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:29.829398 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-49b4bb9a-b1d1-4151-b39d-624fe66757da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:29.903373 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1226/2450] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:26:29.905267 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb15ea-a839-4ca8-8010-866ce287bd28 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:29.905566 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb15ea-a839-4ca8-8010-866ce287bd28 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:29.905852 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb15ea-a839-4ca8-8010-866ce287bd28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:29.926045 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-e1fb15ea-a839-4ca8-8010-866ce287bd28 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:29.927313 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e1fb15ea-a839-4ca8-8010-866ce287bd28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:29.935568 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1225/2451] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:26:29.939849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b4bb9a-b1d1-4151-b39d-624fe66757da None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:29.950304 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b4bb9a-b1d1-4151-b39d-624fe66757da None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:29.955416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed760e3-8887-4632-ae5a-3b2c2b083faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:29.956103 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed760e3-8887-4632-ae5a-3b2c2b083faf admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:29.956384 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed760e3-8887-4632-ae5a-3b2c2b083faf admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:29.956656 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed760e3-8887-4632-ae5a-3b2c2b083faf admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:29.957502 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed760e3-8887-4632-ae5a-3b2c2b083faf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:26:29.957805 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed760e3-8887-4632-ae5a-3b2c2b083faf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:29.960079 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed760e3-8887-4632-ae5a-3b2c2b083faf admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:29.972416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1227/2452] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:26:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:26:29.977441 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb15ea-a839-4ca8-8010-866ce287bd28 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:29.987354 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb15ea-a839-4ca8-8010-866ce287bd28 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:29.992266 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652a10fa-2e2c-4c16-9b6b-19bf2fa0fcbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:29.992924 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a10fa-2e2c-4c16-9b6b-19bf2fa0fcbe admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:29.993242 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a10fa-2e2c-4c16-9b6b-19bf2fa0fcbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:29.993517 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a10fa-2e2c-4c16-9b6b-19bf2fa0fcbe admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:29.994618 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652a10fa-2e2c-4c16-9b6b-19bf2fa0fcbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:29.996364 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652a10fa-2e2c-4c16-9b6b-19bf2fa0fcbe admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:30.024574 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1226/2453] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:26:29 2022] POST /identity/v3/projects => generated 367 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:26:30.028754 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed760e3-8887-4632-ae5a-3b2c2b083faf admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:30.039754 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652a10fa-2e2c-4c16-9b6b-19bf2fa0fcbe admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:30.042339 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-652a10fa-2e2c-4c16-9b6b-19bf2fa0fcbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:30.043888 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed760e3-8887-4632-ae5a-3b2c2b083faf admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:30.092927 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:30.110374 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4833048-5213-42d3-b134-a72778ced666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:30.111322 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4833048-5213-42d3-b134-a72778ced666 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:30.111612 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4833048-5213-42d3-b134-a72778ced666 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:30.111923 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4833048-5213-42d3-b134-a72778ced666 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:30.112622 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4833048-5213-42d3-b134-a72778ced666 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:30.114585 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4833048-5213-42d3-b134-a72778ced666 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:30.119811 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652a10fa-2e2c-4c16-9b6b-19bf2fa0fcbe admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:30.124648 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4750f935-fe71-424a-a706-38693be5f178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:30.125564 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750f935-fe71-424a-a706-38693be5f178 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:30.125848 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750f935-fe71-424a-a706-38693be5f178 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:30.126128 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750f935-fe71-424a-a706-38693be5f178 admin admin] PATH_INFO: `/v3/projects/2a58079f42e946608627ec1a3e7251a3` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:30.127335 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4750f935-fe71-424a-a706-38693be5f178 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a58079f42e946608627ec1a3e7251a3)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:30.129280 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4750f935-fe71-424a-a706-38693be5f178 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:30.134851 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1228/2454] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:26:30 2022] POST /identity/v3/users => generated 284 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:26:30.139032 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4833048-5213-42d3-b134-a72778ced666 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:30.148985 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4833048-5213-42d3-b134-a72778ced666 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:30.153698 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1c643f-a9cf-4f84-8a30-73606ad2b488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:30.154339 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c643f-a9cf-4f84-8a30-73606ad2b488 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:30.154620 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c643f-a9cf-4f84-8a30-73606ad2b488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:30.154897 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c643f-a9cf-4f84-8a30-73606ad2b488 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:30.155614 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1c643f-a9cf-4f84-8a30-73606ad2b488 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:30.157251 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1c643f-a9cf-4f84-8a30-73606ad2b488 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:30.168514 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1229/2455] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:26:30.172435 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1c643f-a9cf-4f84-8a30-73606ad2b488 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:30.178730 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4750f935-fe71-424a-a706-38693be5f178 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a58079f42e946608627ec1a3e7251a3', 'request_id': 'req-4750f935-fe71-424a-a706-38693be5f178'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:30.182046 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1c643f-a9cf-4f84-8a30-73606ad2b488 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:30.184347 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4750f935-fe71-424a-a706-38693be5f178 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a58079f42e946608627ec1a3e7251a3', 'request_id': 'req-4750f935-fe71-424a-a706-38693be5f178'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:30.190902 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4750f935-fe71-424a-a706-38693be5f178 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a58079f42e946608627ec1a3e7251a3', 'request_id': 'req-4750f935-fe71-424a-a706-38693be5f178'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:30.206570 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4750f935-fe71-424a-a706-38693be5f178 admin admin] The token cache is being invalidate because project 2a58079f42e946608627ec1a3e7251a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:26:30.206868 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4750f935-fe71-424a-a706-38693be5f178 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:30.207667 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4750f935-fe71-424a-a706-38693be5f178 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:30.210832 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1227/2456] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:26:30 2022] DELETE /identity/v3/projects/2a58079f42e946608627ec1a3e7251a3 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:26:30.216002 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d77e88-7c27-4bd9-aba7-0f7bfdb4086a None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:30.216350 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d77e88-7c27-4bd9-aba7-0f7bfdb4086a None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:30.216764 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d77e88-7c27-4bd9-aba7-0f7bfdb4086a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:30.240621 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-57d77e88-7c27-4bd9-aba7-0f7bfdb4086a None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:30.242037 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-57d77e88-7c27-4bd9-aba7-0f7bfdb4086a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:30.251673 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23977f-2389-4dfe-b8e4-5ded31dc1070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:30.252566 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23977f-2389-4dfe-b8e4-5ded31dc1070 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:30.252923 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23977f-2389-4dfe-b8e4-5ded31dc1070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:30.253297 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23977f-2389-4dfe-b8e4-5ded31dc1070 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:30.254035 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d23977f-2389-4dfe-b8e4-5ded31dc1070 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:30.256296 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d23977f-2389-4dfe-b8e4-5ded31dc1070 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:30.268650 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1230/2457] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:26:30.273808 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23977f-2389-4dfe-b8e4-5ded31dc1070 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:30.276115 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6d23977f-2389-4dfe-b8e4-5ded31dc1070 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:30.281431 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:30.292717 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23977f-2389-4dfe-b8e4-5ded31dc1070 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:30.298368 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e988c0b-7464-4ca3-ba18-e34ad7e15f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:30.299265 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e988c0b-7464-4ca3-ba18-e34ad7e15f38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:30.299542 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e988c0b-7464-4ca3-ba18-e34ad7e15f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:30.299818 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e988c0b-7464-4ca3-ba18-e34ad7e15f38 admin admin] PATH_INFO: `/v3/projects/c921ed64bc6e4836906281802739441d/users/325dc0af022b46b9bfb88197bd7c1344/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:30.322008 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1228/2458] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:26:30.322215 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e988c0b-7464-4ca3-ba18-e34ad7e15f38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c921ed64bc6e4836906281802739441d, user_id=325dc0af022b46b9bfb88197bd7c1344, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:30.322215 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e988c0b-7464-4ca3-ba18-e34ad7e15f38 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:30.323839 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff5d667-52ce-49ed-81d5-e3f5ddbe7c6d None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:30.324211 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff5d667-52ce-49ed-81d5-e3f5ddbe7c6d None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:30.324493 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff5d667-52ce-49ed-81d5-e3f5ddbe7c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:30.337619 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1231/2459] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:26:30 2022] PUT /identity/v3/projects/c921ed64bc6e4836906281802739441d/users/325dc0af022b46b9bfb88197bd7c1344/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:30.346703 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-cff5d667-52ce-49ed-81d5-e3f5ddbe7c6d None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:30.347228 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-cff5d667-52ce-49ed-81d5-e3f5ddbe7c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:30.347582 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eafb28-65ea-44ee-9948-ea27923499b5 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:30.347876 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eafb28-65ea-44ee-9948-ea27923499b5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:30.348167 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eafb28-65ea-44ee-9948-ea27923499b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:30.376416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-54eafb28-65ea-44ee-9948-ea27923499b5 None None] MFA Rules not processed for user `325dc0af022b46b9bfb88197bd7c1344`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:30.377689 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-54eafb28-65ea-44ee-9948-ea27923499b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:30.429919 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1229/2460] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:26:30.434092 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff5d667-52ce-49ed-81d5-e3f5ddbe7c6d None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:30.443974 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff5d667-52ce-49ed-81d5-e3f5ddbe7c6d None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:30.451725 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863b846a-d086-48c5-bfbc-dec68ed5903f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:30.453101 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b846a-d086-48c5-bfbc-dec68ed5903f admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:30.453485 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b846a-d086-48c5-bfbc-dec68ed5903f admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:30.454198 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b846a-d086-48c5-bfbc-dec68ed5903f admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:30.455676 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863b846a-d086-48c5-bfbc-dec68ed5903f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:26:30.456014 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863b846a-d086-48c5-bfbc-dec68ed5903f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:30.458963 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863b846a-d086-48c5-bfbc-dec68ed5903f admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:30.459608 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1232/2461] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:30 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:26:30.465517 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eafb28-65ea-44ee-9948-ea27923499b5 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:30.478117 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1230/2462] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:26:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:26:30.478287 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eafb28-65ea-44ee-9948-ea27923499b5 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:30.479085 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863b846a-d086-48c5-bfbc-dec68ed5903f admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:30.482831 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca0037-95f4-434e-830c-d6a704814f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:30.483498 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca0037-95f4-434e-830c-d6a704814f32 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:30.483825 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca0037-95f4-434e-830c-d6a704814f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:30.484802 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca0037-95f4-434e-830c-d6a704814f32 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:30.485115 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdca0037-95f4-434e-830c-d6a704814f32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:30.488233 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdca0037-95f4-434e-830c-d6a704814f32 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:30.489966 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863b846a-d086-48c5-bfbc-dec68ed5903f admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:30.495721 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c81a988-41dc-4948-97c5-672bf4cff1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:30.496554 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81a988-41dc-4948-97c5-672bf4cff1ba admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:30.496703 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81a988-41dc-4948-97c5-672bf4cff1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:30.496961 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81a988-41dc-4948-97c5-672bf4cff1ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:30.497981 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c81a988-41dc-4948-97c5-672bf4cff1ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:30.499787 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c81a988-41dc-4948-97c5-672bf4cff1ba admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:30.522044 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1233/2463] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:26:30 2022] POST /identity/v3/projects => generated 367 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:26:30.525711 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca0037-95f4-434e-830c-d6a704814f32 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:30.533760 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1231/2464] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:26:30 2022] POST /identity/v3/projects => generated 369 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:26:30.533940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:30.535286 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c81a988-41dc-4948-97c5-672bf4cff1ba admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:30.635817 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca0037-95f4-434e-830c-d6a704814f32 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:30.636291 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c81a988-41dc-4948-97c5-672bf4cff1ba admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:30.641215 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7657e021-4713-4de6-9a45-bce0913b34ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:30.641903 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7657e021-4713-4de6-9a45-bce0913b34ec admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:30.642193 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7657e021-4713-4de6-9a45-bce0913b34ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:30.642475 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7657e021-4713-4de6-9a45-bce0913b34ec admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:30.643436 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7657e021-4713-4de6-9a45-bce0913b34ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:30.643873 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8600f06-2b76-46c1-a545-98a3a5e31eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:30.644649 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8600f06-2b76-46c1-a545-98a3a5e31eb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:30.645198 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8600f06-2b76-46c1-a545-98a3a5e31eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:30.645571 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7657e021-4713-4de6-9a45-bce0913b34ec admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:30.645792 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8600f06-2b76-46c1-a545-98a3a5e31eb0 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:30.646730 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8600f06-2b76-46c1-a545-98a3a5e31eb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:30.650459 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8600f06-2b76-46c1-a545-98a3a5e31eb0 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:30.673082 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1234/2465] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:26:30 2022] POST /identity/v3/users => generated 290 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:26:30.675442 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1232/2466] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:26:30 2022] POST /identity/v3/users => generated 285 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:26:30.675685 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7657e021-4713-4de6-9a45-bce0913b34ec admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:30.680131 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8600f06-2b76-46c1-a545-98a3a5e31eb0 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:30.687942 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7657e021-4713-4de6-9a45-bce0913b34ec admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:30.690466 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8600f06-2b76-46c1-a545-98a3a5e31eb0 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:30.693292 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22604f9-d888-403a-b80e-1adeb663b208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:30.693963 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22604f9-d888-403a-b80e-1adeb663b208 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:30.694250 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22604f9-d888-403a-b80e-1adeb663b208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:30.694531 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22604f9-d888-403a-b80e-1adeb663b208 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:30.695284 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22604f9-d888-403a-b80e-1adeb663b208 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:30.696315 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff1e139-be31-421c-8e91-0856c3206bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:30.696872 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22604f9-d888-403a-b80e-1adeb663b208 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:30.696975 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1e139-be31-421c-8e91-0856c3206bb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:30.697283 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1e139-be31-421c-8e91-0856c3206bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:30.697573 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1e139-be31-421c-8e91-0856c3206bb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:30.698462 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff1e139-be31-421c-8e91-0856c3206bb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:30.700429 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff1e139-be31-421c-8e91-0856c3206bb1 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:30.706962 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1235/2467] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:26:30.711182 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1233/2468] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:26:30.711571 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22604f9-d888-403a-b80e-1adeb663b208 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:30.714866 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff1e139-be31-421c-8e91-0856c3206bb1 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:30.721622 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22604f9-d888-403a-b80e-1adeb663b208 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:30.725392 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff1e139-be31-421c-8e91-0856c3206bb1 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:30.726463 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bdf55c-3fc7-48d2-8f2d-5e87d1181aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:30.727337 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bdf55c-3fc7-48d2-8f2d-5e87d1181aef admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:30.727629 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bdf55c-3fc7-48d2-8f2d-5e87d1181aef admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:30.727911 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bdf55c-3fc7-48d2-8f2d-5e87d1181aef admin admin] PATH_INFO: `/v3/projects/14cf7bb7e4484ad18abd5f822cbdcc55/users/a2b637ca441b4f759f0442b87dd664cc/roles/f36e053b442540f08041ac7878760f65` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:30.730084 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f385deab-8a73-4000-9294-51dc9dcac6c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:30.730731 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f385deab-8a73-4000-9294-51dc9dcac6c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:30.731022 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f385deab-8a73-4000-9294-51dc9dcac6c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:30.731302 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f385deab-8a73-4000-9294-51dc9dcac6c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:30.732059 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f385deab-8a73-4000-9294-51dc9dcac6c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:30.733704 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f385deab-8a73-4000-9294-51dc9dcac6c4 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:30.743586 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bdf55c-3fc7-48d2-8f2d-5e87d1181aef admin admin] RBAC: Authorizing `identity:create_grant(project_id=14cf7bb7e4484ad18abd5f822cbdcc55, user_id=a2b637ca441b4f759f0442b87dd664cc, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:30.744271 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1234/2469] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:26:30.746637 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bdf55c-3fc7-48d2-8f2d-5e87d1181aef admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:30.748426 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f385deab-8a73-4000-9294-51dc9dcac6c4 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:30.760409 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f385deab-8a73-4000-9294-51dc9dcac6c4 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:30.761758 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1236/2470] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:26:30 2022] PUT /identity/v3/projects/14cf7bb7e4484ad18abd5f822cbdcc55/users/a2b637ca441b4f759f0442b87dd664cc/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:26:30.765528 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bdf55c-3fc7-48d2-8f2d-5e87d1181aef admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:30.769803 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:30.827738 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4947e4e-2d3d-4375-bc47-a42887658813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:30.828652 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4947e4e-2d3d-4375-bc47-a42887658813 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:30.829192 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4947e4e-2d3d-4375-bc47-a42887658813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:30.829395 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4947e4e-2d3d-4375-bc47-a42887658813 admin admin] PATH_INFO: `/v3/projects/49bd928841d043708a3f54f4525fcdc7/users/0cab15d48dbc47efb36822ccf4aabe84/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:30.836758 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bdf55c-3fc7-48d2-8f2d-5e87d1181aef admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:30.842780 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4fc8b6-7ab7-4e37-8f0c-17dd2e2bfc3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:30.843866 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4fc8b6-7ab7-4e37-8f0c-17dd2e2bfc3b admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:30.844164 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4fc8b6-7ab7-4e37-8f0c-17dd2e2bfc3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:30.844604 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4fc8b6-7ab7-4e37-8f0c-17dd2e2bfc3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:30.845501 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4fc8b6-7ab7-4e37-8f0c-17dd2e2bfc3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:30.847472 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4fc8b6-7ab7-4e37-8f0c-17dd2e2bfc3b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:30.850656 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4947e4e-2d3d-4375-bc47-a42887658813 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49bd928841d043708a3f54f4525fcdc7, user_id=0cab15d48dbc47efb36822ccf4aabe84, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:30.853280 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4947e4e-2d3d-4375-bc47-a42887658813 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:30.861082 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1237/2471] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:26:30.866452 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4fc8b6-7ab7-4e37-8f0c-17dd2e2bfc3b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:30.879127 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1235/2472] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:26:30 2022] PUT /identity/v3/projects/49bd928841d043708a3f54f4525fcdc7/users/0cab15d48dbc47efb36822ccf4aabe84/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:26:30.879465 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4fc8b6-7ab7-4e37-8f0c-17dd2e2bfc3b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:30.880900 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a295e1d-f8eb-4190-ad13-e70354530f20 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:30.881242 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a295e1d-f8eb-4190-ad13-e70354530f20 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:30.881526 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a295e1d-f8eb-4190-ad13-e70354530f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:30.910456 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-7a295e1d-f8eb-4190-ad13-e70354530f20 None None] MFA Rules not processed for user `0cab15d48dbc47efb36822ccf4aabe84`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:30.912572 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7a295e1d-f8eb-4190-ad13-e70354530f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:30.937461 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:30.954763 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61662e96-b752-498d-9171-25f83dc910c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:30.955597 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61662e96-b752-498d-9171-25f83dc910c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:30.956299 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61662e96-b752-498d-9171-25f83dc910c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:30.956601 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61662e96-b752-498d-9171-25f83dc910c5 admin admin] PATH_INFO: `/v3/projects/14cf7bb7e4484ad18abd5f822cbdcc55/users/a2b637ca441b4f759f0442b87dd664cc/roles/f36e053b442540f08041ac7878760f65` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:30.959195 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61662e96-b752-498d-9171-25f83dc910c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14cf7bb7e4484ad18abd5f822cbdcc55, user_id=a2b637ca441b4f759f0442b87dd664cc, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:30.962751 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61662e96-b752-498d-9171-25f83dc910c5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:30.982146 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1238/2473] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:26:30 2022] PUT /identity/v3/projects/14cf7bb7e4484ad18abd5f822cbdcc55/users/a2b637ca441b4f759f0442b87dd664cc/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:30.994367 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7871b1-accd-4cb1-9576-68b8758d9878 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:30.994799 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7871b1-accd-4cb1-9576-68b8758d9878 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:30.995185 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7871b1-accd-4cb1-9576-68b8758d9878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:30.996441 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1236/2474] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:30 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:26:31.000859 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a295e1d-f8eb-4190-ad13-e70354530f20 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:31.011786 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a295e1d-f8eb-4190-ad13-e70354530f20 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:31.028170 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8f7871b1-accd-4cb1-9576-68b8758d9878 None None] MFA Rules not processed for user `a2b637ca441b4f759f0442b87dd664cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:31.029312 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8f7871b1-accd-4cb1-9576-68b8758d9878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:31.078717 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a128cffb-f8d3-465f-8e86-a931182bf4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:31.079389 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a128cffb-f8d3-465f-8e86-a931182bf4f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:31.079671 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a128cffb-f8d3-465f-8e86-a931182bf4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:31.079947 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a128cffb-f8d3-465f-8e86-a931182bf4f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:31.080638 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a128cffb-f8d3-465f-8e86-a931182bf4f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:31.082590 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a128cffb-f8d3-465f-8e86-a931182bf4f2 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:31.117638 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1237/2475] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:26:30 2022] POST /identity/v3/projects => generated 369 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:26:31.121659 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a128cffb-f8d3-465f-8e86-a931182bf4f2 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:31.129156 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1239/2476] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:30 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:26:31.137316 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7871b1-accd-4cb1-9576-68b8758d9878 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:31.138900 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8f7871b1-accd-4cb1-9576-68b8758d9878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:31.142396 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:31.213210 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a128cffb-f8d3-465f-8e86-a931182bf4f2 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:31.221307 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e5b4b2-210e-4063-9d02-9a53aaa740e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:31.221804 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e5b4b2-210e-4063-9d02-9a53aaa740e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:31.222095 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e5b4b2-210e-4063-9d02-9a53aaa740e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:31.222403 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e5b4b2-210e-4063-9d02-9a53aaa740e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:31.223260 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e5b4b2-210e-4063-9d02-9a53aaa740e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:31.225995 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e5b4b2-210e-4063-9d02-9a53aaa740e7 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:31.239929 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7871b1-accd-4cb1-9576-68b8758d9878 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:31.257246 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1238/2477] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:26:31 2022] POST /identity/v3/users => generated 291 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:26:31.261973 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e5b4b2-210e-4063-9d02-9a53aaa740e7 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:31.276403 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e5b4b2-210e-4063-9d02-9a53aaa740e7 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:31.281906 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b79e3b9-af9e-416e-b0d3-ba78cd61a1b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:31.282572 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79e3b9-af9e-416e-b0d3-ba78cd61a1b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:31.283243 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79e3b9-af9e-416e-b0d3-ba78cd61a1b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:31.283534 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79e3b9-af9e-416e-b0d3-ba78cd61a1b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:31.284261 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b79e3b9-af9e-416e-b0d3-ba78cd61a1b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:31.285907 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b79e3b9-af9e-416e-b0d3-ba78cd61a1b1 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:31.297311 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1239/2478] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:26:31 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:26:31.301333 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b79e3b9-af9e-416e-b0d3-ba78cd61a1b1 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:31.309394 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7952b3-9638-4829-a59f-d3ba882547d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:31.310053 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7952b3-9638-4829-a59f-d3ba882547d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:31.310769 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7952b3-9638-4829-a59f-d3ba882547d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:31.310769 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7952b3-9638-4829-a59f-d3ba882547d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:31.314741 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b79e3b9-af9e-416e-b0d3-ba78cd61a1b1 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:31.319834 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f255ed83-4c06-47ad-b9bf-bc01a536d8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:31.320720 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255ed83-4c06-47ad-b9bf-bc01a536d8ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:31.321031 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255ed83-4c06-47ad-b9bf-bc01a536d8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:31.321346 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255ed83-4c06-47ad-b9bf-bc01a536d8ab admin admin] PATH_INFO: `/v3/projects/7e8e7c829d814f62bdc33637aba77ae5/users/25dc4987d751446c91ffecd953dae7d3/roles/f36e053b442540f08041ac7878760f65` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:31.337366 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f255ed83-4c06-47ad-b9bf-bc01a536d8ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e8e7c829d814f62bdc33637aba77ae5, user_id=25dc4987d751446c91ffecd953dae7d3, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:31.339970 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f255ed83-4c06-47ad-b9bf-bc01a536d8ab admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:31.356735 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:31.357617 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1240/2479] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:26:31 2022] PUT /identity/v3/projects/7e8e7c829d814f62bdc33637aba77ae5/users/25dc4987d751446c91ffecd953dae7d3/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:26:31.362596 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f255ed83-4c06-47ad-b9bf-bc01a536d8ab admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:31.379834 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7952b3-9638-4829-a59f-d3ba882547d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:31.381539 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7952b3-9638-4829-a59f-d3ba882547d3 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:31.391851 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1240/2480] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:26:31 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:26:31.428617 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f255ed83-4c06-47ad-b9bf-bc01a536d8ab admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:31.433614 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142f9181-65bd-4bee-ae74-2a9fd5e4cc63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:31.434454 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f9181-65bd-4bee-ae74-2a9fd5e4cc63 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:31.434746 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f9181-65bd-4bee-ae74-2a9fd5e4cc63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:31.435057 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f9181-65bd-4bee-ae74-2a9fd5e4cc63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:31.435793 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142f9181-65bd-4bee-ae74-2a9fd5e4cc63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:31.437514 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142f9181-65bd-4bee-ae74-2a9fd5e4cc63 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:31.452553 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1241/2481] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:26:31 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:26:31.457347 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7952b3-9638-4829-a59f-d3ba882547d3 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:31.471946 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7952b3-9638-4829-a59f-d3ba882547d3 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:31.477856 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f560da-c61b-4b85-ba0b-b65a72e4a831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:31.478997 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f560da-c61b-4b85-ba0b-b65a72e4a831 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:31.479507 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f560da-c61b-4b85-ba0b-b65a72e4a831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:31.479942 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f560da-c61b-4b85-ba0b-b65a72e4a831 admin admin] PATH_INFO: `/v3/projects/7e8e7c829d814f62bdc33637aba77ae5/users/25dc4987d751446c91ffecd953dae7d3/roles/f36e053b442540f08041ac7878760f65` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:31.482188 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f560da-c61b-4b85-ba0b-b65a72e4a831 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e8e7c829d814f62bdc33637aba77ae5, user_id=25dc4987d751446c91ffecd953dae7d3, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:31.485556 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f560da-c61b-4b85-ba0b-b65a72e4a831 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:31.496457 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1241/2482] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:26:31 2022] PUT /identity/v3/projects/7e8e7c829d814f62bdc33637aba77ae5/users/25dc4987d751446c91ffecd953dae7d3/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:31.563417 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f002c5d6-0353-4f05-b3ac-ae77a19fe846 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:31.563710 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f002c5d6-0353-4f05-b3ac-ae77a19fe846 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:31.563999 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f002c5d6-0353-4f05-b3ac-ae77a19fe846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:31.598072 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-f002c5d6-0353-4f05-b3ac-ae77a19fe846 None None] MFA Rules not processed for user `25dc4987d751446c91ffecd953dae7d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:31.599442 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f002c5d6-0353-4f05-b3ac-ae77a19fe846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:31.651721 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:31.677724 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1242/2483] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:31 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:26:31.715602 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f560da-c61b-4b85-ba0b-b65a72e4a831 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:31.777405 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f560da-c61b-4b85-ba0b-b65a72e4a831 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:31.780958 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437c8d7d-0c42-4b1b-89b5-a51b1053b09b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:31.781647 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c8d7d-0c42-4b1b-89b5-a51b1053b09b service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:31.782028 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c8d7d-0c42-4b1b-89b5-a51b1053b09b service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:31.782225 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c8d7d-0c42-4b1b-89b5-a51b1053b09b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:31.792154 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437c8d7d-0c42-4b1b-89b5-a51b1053b09b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:31.793684 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437c8d7d-0c42-4b1b-89b5-a51b1053b09b service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:31.803803 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1242/2484] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:26:31 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:26:31.847737 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f002c5d6-0353-4f05-b3ac-ae77a19fe846 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:31.849937 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f002c5d6-0353-4f05-b3ac-ae77a19fe846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:31.863404 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f002c5d6-0353-4f05-b3ac-ae77a19fe846 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:31.872988 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72bbd99-9c4a-4aa7-b69c-768c7c487279 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:31.873688 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72bbd99-9c4a-4aa7-b69c-768c7c487279 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:31.873981 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72bbd99-9c4a-4aa7-b69c-768c7c487279 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:31.874299 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72bbd99-9c4a-4aa7-b69c-768c7c487279 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:31.939388 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72bbd99-9c4a-4aa7-b69c-768c7c487279 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:31.940874 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72bbd99-9c4a-4aa7-b69c-768c7c487279 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:31.954615 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1243/2485] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:26:31 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:26:32.861734 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437c8d7d-0c42-4b1b-89b5-a51b1053b09b service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:32.940101 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437c8d7d-0c42-4b1b-89b5-a51b1053b09b service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:32.945695 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2daa74-2452-41b8-98e5-fcff57b8e013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:32.946578 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2daa74-2452-41b8-98e5-fcff57b8e013 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:32.946872 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2daa74-2452-41b8-98e5-fcff57b8e013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:32.947152 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2daa74-2452-41b8-98e5-fcff57b8e013 admin admin] PATH_INFO: `/v3/users/325dc0af022b46b9bfb88197bd7c1344` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:32.948355 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2daa74-2452-41b8-98e5-fcff57b8e013 admin admin] RBAC: Authorizing `identity:delete_user(user_id=325dc0af022b46b9bfb88197bd7c1344)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:32.950354 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2daa74-2452-41b8-98e5-fcff57b8e013 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:33.002223 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-be2daa74-2452-41b8-98e5-fcff57b8e013 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '325dc0af022b46b9bfb88197bd7c1344'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:33.002940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-be2daa74-2452-41b8-98e5-fcff57b8e013 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '325dc0af022b46b9bfb88197bd7c1344'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:33.011655 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-be2daa74-2452-41b8-98e5-fcff57b8e013 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '325dc0af022b46b9bfb88197bd7c1344'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:33.024905 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:33.026434 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-be2daa74-2452-41b8-98e5-fcff57b8e013 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '325dc0af022b46b9bfb88197bd7c1344'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:33.027508 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-be2daa74-2452-41b8-98e5-fcff57b8e013 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '325dc0af022b46b9bfb88197bd7c1344'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:33.043777 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1243/2486] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:26:32 2022] DELETE /identity/v3/users/325dc0af022b46b9bfb88197bd7c1344 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:33.048422 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a05810-db05-432d-9d8f-b563532e9db6 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:33.048943 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a05810-db05-432d-9d8f-b563532e9db6 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:33.049070 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a05810-db05-432d-9d8f-b563532e9db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:33.065673 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-b7a05810-db05-432d-9d8f-b563532e9db6 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:33.066900 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b7a05810-db05-432d-9d8f-b563532e9db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:33.127686 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1244/2487] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:26:33.137941 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2daa74-2452-41b8-98e5-fcff57b8e013 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:33.139377 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-be2daa74-2452-41b8-98e5-fcff57b8e013 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:33.196879 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2daa74-2452-41b8-98e5-fcff57b8e013 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:33.204830 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24ff09-bb17-49ad-a331-c8df4575cac9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:33.205527 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24ff09-bb17-49ad-a331-c8df4575cac9 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:33.205810 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24ff09-bb17-49ad-a331-c8df4575cac9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:33.206216 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24ff09-bb17-49ad-a331-c8df4575cac9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:33.219434 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e24ff09-bb17-49ad-a331-c8df4575cac9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:33.220912 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e24ff09-bb17-49ad-a331-c8df4575cac9 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:33.231477 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1244/2488] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:26:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:26:33.526225 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a05810-db05-432d-9d8f-b563532e9db6 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:33.528202 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b7a05810-db05-432d-9d8f-b563532e9db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:33.543824 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a05810-db05-432d-9d8f-b563532e9db6 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:33.557001 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3587dbc-2bcd-427c-b992-6a0a68ebef84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:33.558975 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3587dbc-2bcd-427c-b992-6a0a68ebef84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:33.559273 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3587dbc-2bcd-427c-b992-6a0a68ebef84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:33.559572 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3587dbc-2bcd-427c-b992-6a0a68ebef84 admin admin] PATH_INFO: `/v3/projects/c921ed64bc6e4836906281802739441d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:33.560959 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3587dbc-2bcd-427c-b992-6a0a68ebef84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c921ed64bc6e4836906281802739441d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:33.563032 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3587dbc-2bcd-427c-b992-6a0a68ebef84 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:33.619459 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b3587dbc-2bcd-427c-b992-6a0a68ebef84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c921ed64bc6e4836906281802739441d', 'request_id': 'req-b3587dbc-2bcd-427c-b992-6a0a68ebef84'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:33.625834 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b3587dbc-2bcd-427c-b992-6a0a68ebef84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c921ed64bc6e4836906281802739441d', 'request_id': 'req-b3587dbc-2bcd-427c-b992-6a0a68ebef84'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:33.632166 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b3587dbc-2bcd-427c-b992-6a0a68ebef84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c921ed64bc6e4836906281802739441d', 'request_id': 'req-b3587dbc-2bcd-427c-b992-6a0a68ebef84'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:33.646701 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b3587dbc-2bcd-427c-b992-6a0a68ebef84 admin admin] The token cache is being invalidate because project c921ed64bc6e4836906281802739441d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:26:33.646903 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b3587dbc-2bcd-427c-b992-6a0a68ebef84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:33.647642 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b3587dbc-2bcd-427c-b992-6a0a68ebef84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:33.649775 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1245/2489] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:26:33 2022] DELETE /identity/v3/projects/c921ed64bc6e4836906281802739441d => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:33.653959 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24ff09-bb17-49ad-a331-c8df4575cac9 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:33.665385 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24ff09-bb17-49ad-a331-c8df4575cac9 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:33.750476 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4c5515-b09f-4c8c-8145-3c0af73a41de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:33.751518 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4c5515-b09f-4c8c-8145-3c0af73a41de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:33.751630 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4c5515-b09f-4c8c-8145-3c0af73a41de admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:33.752044 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4c5515-b09f-4c8c-8145-3c0af73a41de admin admin] PATH_INFO: `/v3/users/a2b637ca441b4f759f0442b87dd664cc` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:33.753279 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4c5515-b09f-4c8c-8145-3c0af73a41de admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2b637ca441b4f759f0442b87dd664cc)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:33.755887 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4c5515-b09f-4c8c-8145-3c0af73a41de admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:33.806374 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3f4c5515-b09f-4c8c-8145-3c0af73a41de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2b637ca441b4f759f0442b87dd664cc'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:33.807037 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3f4c5515-b09f-4c8c-8145-3c0af73a41de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2b637ca441b4f759f0442b87dd664cc'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:33.814874 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3f4c5515-b09f-4c8c-8145-3c0af73a41de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2b637ca441b4f759f0442b87dd664cc'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:33.827983 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3f4c5515-b09f-4c8c-8145-3c0af73a41de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2b637ca441b4f759f0442b87dd664cc'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:33.828648 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3f4c5515-b09f-4c8c-8145-3c0af73a41de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2b637ca441b4f759f0442b87dd664cc'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:33.842649 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1245/2490] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:26:33 2022] DELETE /identity/v3/users/a2b637ca441b4f759f0442b87dd664cc => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:34.119535 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3587dbc-2bcd-427c-b992-6a0a68ebef84 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:34.121412 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b3587dbc-2bcd-427c-b992-6a0a68ebef84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:34.137492 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3587dbc-2bcd-427c-b992-6a0a68ebef84 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:34.143333 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:34.209790 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be19966-7f2d-480b-8438-77502fe4ee22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:34.210827 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be19966-7f2d-480b-8438-77502fe4ee22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:34.211146 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be19966-7f2d-480b-8438-77502fe4ee22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:34.211453 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be19966-7f2d-480b-8438-77502fe4ee22 admin admin] PATH_INFO: `/v3/projects/14cf7bb7e4484ad18abd5f822cbdcc55` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:34.212785 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be19966-7f2d-480b-8438-77502fe4ee22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14cf7bb7e4484ad18abd5f822cbdcc55)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:34.215101 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be19966-7f2d-480b-8438-77502fe4ee22 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:34.261776 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-5be19966-7f2d-480b-8438-77502fe4ee22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14cf7bb7e4484ad18abd5f822cbdcc55', 'request_id': 'req-5be19966-7f2d-480b-8438-77502fe4ee22'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:34.267286 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-5be19966-7f2d-480b-8438-77502fe4ee22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14cf7bb7e4484ad18abd5f822cbdcc55', 'request_id': 'req-5be19966-7f2d-480b-8438-77502fe4ee22'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:34.273610 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-5be19966-7f2d-480b-8438-77502fe4ee22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14cf7bb7e4484ad18abd5f822cbdcc55', 'request_id': 'req-5be19966-7f2d-480b-8438-77502fe4ee22'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:34.285430 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-5be19966-7f2d-480b-8438-77502fe4ee22 admin admin] The token cache is being invalidate because project 14cf7bb7e4484ad18abd5f822cbdcc55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:26:34.285765 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-5be19966-7f2d-480b-8438-77502fe4ee22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:34.286466 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-5be19966-7f2d-480b-8438-77502fe4ee22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:34.288001 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1246/2491] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:26:34 2022] DELETE /identity/v3/projects/14cf7bb7e4484ad18abd5f822cbdcc55 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:34.293031 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1962bf4-0687-4a9e-b357-e5fe60ed4fe0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:34.293548 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1962bf4-0687-4a9e-b357-e5fe60ed4fe0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:34.293638 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1962bf4-0687-4a9e-b357-e5fe60ed4fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:34.313346 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f1962bf4-0687-4a9e-b357-e5fe60ed4fe0 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:34.314569 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f1962bf4-0687-4a9e-b357-e5fe60ed4fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:34.319396 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:34.378130 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1246/2492] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:26:34.386879 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc6e0a-867b-49f8-be3f-5e64ffa24104 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:34.387247 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc6e0a-867b-49f8-be3f-5e64ffa24104 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:34.387600 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc6e0a-867b-49f8-be3f-5e64ffa24104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:34.406136 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-69bc6e0a-867b-49f8-be3f-5e64ffa24104 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:34.407355 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-69bc6e0a-867b-49f8-be3f-5e64ffa24104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:34.415332 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1247/2493] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:26:34.419547 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1962bf4-0687-4a9e-b357-e5fe60ed4fe0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:34.430285 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1962bf4-0687-4a9e-b357-e5fe60ed4fe0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:34.435496 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06937d39-3e73-4e0d-b535-0174130f2671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:34.436154 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06937d39-3e73-4e0d-b535-0174130f2671 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:34.436437 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06937d39-3e73-4e0d-b535-0174130f2671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:34.436722 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06937d39-3e73-4e0d-b535-0174130f2671 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:34.437505 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06937d39-3e73-4e0d-b535-0174130f2671 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:26:34.437813 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06937d39-3e73-4e0d-b535-0174130f2671 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:34.439418 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06937d39-3e73-4e0d-b535-0174130f2671 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:34.451998 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1247/2494] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:26:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:26:34.456189 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc6e0a-867b-49f8-be3f-5e64ffa24104 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:34.465805 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc6e0a-867b-49f8-be3f-5e64ffa24104 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:34.470674 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4306ec-7e43-4794-ae86-0fd64b664739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:34.471327 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4306ec-7e43-4794-ae86-0fd64b664739 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:34.471616 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4306ec-7e43-4794-ae86-0fd64b664739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:34.471900 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4306ec-7e43-4794-ae86-0fd64b664739 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:34.472588 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4306ec-7e43-4794-ae86-0fd64b664739 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:34.474589 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4306ec-7e43-4794-ae86-0fd64b664739 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:34.502451 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1248/2495] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:26:34 2022] POST /identity/v3/projects => generated 393 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:26:34.503967 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06937d39-3e73-4e0d-b535-0174130f2671 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:34.514578 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06937d39-3e73-4e0d-b535-0174130f2671 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:34.570228 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcaef0f-fbfc-45f8-a7f0-8ede0979e1f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:34.570891 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcaef0f-fbfc-45f8-a7f0-8ede0979e1f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:34.571178 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcaef0f-fbfc-45f8-a7f0-8ede0979e1f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:34.571457 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcaef0f-fbfc-45f8-a7f0-8ede0979e1f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:34.572134 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcaef0f-fbfc-45f8-a7f0-8ede0979e1f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:34.574076 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcaef0f-fbfc-45f8-a7f0-8ede0979e1f4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:34.593946 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1248/2496] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:26:34 2022] POST /identity/v3/users => generated 297 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:26:34.598103 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4306ec-7e43-4794-ae86-0fd64b664739 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:34.608860 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4306ec-7e43-4794-ae86-0fd64b664739 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:34.613703 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9c99fa-0877-4f4e-8087-29c2657c69fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:34.614449 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c99fa-0877-4f4e-8087-29c2657c69fa admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:34.615261 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c99fa-0877-4f4e-8087-29c2657c69fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:34.615543 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c99fa-0877-4f4e-8087-29c2657c69fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:34.616275 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9c99fa-0877-4f4e-8087-29c2657c69fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:34.617977 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9c99fa-0877-4f4e-8087-29c2657c69fa admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:34.629176 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1249/2497] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:26:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:26:34.633646 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcaef0f-fbfc-45f8-a7f0-8ede0979e1f4 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:34.643527 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcaef0f-fbfc-45f8-a7f0-8ede0979e1f4 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:34.648413 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2170aec7-f359-4472-ae06-370fbe5a47a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:34.649099 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2170aec7-f359-4472-ae06-370fbe5a47a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:34.649392 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2170aec7-f359-4472-ae06-370fbe5a47a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:34.649677 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2170aec7-f359-4472-ae06-370fbe5a47a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:34.650402 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2170aec7-f359-4472-ae06-370fbe5a47a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:34.652042 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2170aec7-f359-4472-ae06-370fbe5a47a0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:34.662719 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1249/2498] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:26:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:26:34.666718 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9c99fa-0877-4f4e-8087-29c2657c69fa admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:34.676770 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9c99fa-0877-4f4e-8087-29c2657c69fa admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:34.683417 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4556ba1c-7945-4a57-92c5-e9891f2ea66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:34.684864 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4556ba1c-7945-4a57-92c5-e9891f2ea66e admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:34.685338 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4556ba1c-7945-4a57-92c5-e9891f2ea66e admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:34.685768 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4556ba1c-7945-4a57-92c5-e9891f2ea66e admin admin] PATH_INFO: `/v3/projects/c60fbde647bb4e7c9e043fc44bc4cab8/users/e17fd977523f4265b5fd472f8ba2272c/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:34.706192 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4556ba1c-7945-4a57-92c5-e9891f2ea66e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c60fbde647bb4e7c9e043fc44bc4cab8, user_id=e17fd977523f4265b5fd472f8ba2272c, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:34.709117 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4556ba1c-7945-4a57-92c5-e9891f2ea66e admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:34.722328 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1250/2499] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:26:34 2022] PUT /identity/v3/projects/c60fbde647bb4e7c9e043fc44bc4cab8/users/e17fd977523f4265b5fd472f8ba2272c/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:26:34.737069 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c6900-375c-40cd-9e61-14aab87e1da2 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:34.737467 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c6900-375c-40cd-9e61-14aab87e1da2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:34.737750 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c6900-375c-40cd-9e61-14aab87e1da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:34.765760 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a38c6900-375c-40cd-9e61-14aab87e1da2 None None] MFA Rules not processed for user `e17fd977523f4265b5fd472f8ba2272c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:34.766980 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a38c6900-375c-40cd-9e61-14aab87e1da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:34.836056 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1250/2500] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:26:34.840345 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4556ba1c-7945-4a57-92c5-e9891f2ea66e admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:34.886003 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:34.913574 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4556ba1c-7945-4a57-92c5-e9891f2ea66e admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:34.917533 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fd6a7d-72eb-48df-84b4-d5f933949066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:34.918189 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd6a7d-72eb-48df-84b4-d5f933949066 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:34.918473 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd6a7d-72eb-48df-84b4-d5f933949066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:34.918753 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd6a7d-72eb-48df-84b4-d5f933949066 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:34.919437 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fd6a7d-72eb-48df-84b4-d5f933949066 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:34.921360 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fd6a7d-72eb-48df-84b4-d5f933949066 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:34.948328 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1251/2501] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:26:34 2022] POST /identity/v3/projects => generated 393 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:26:34.952363 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c6900-375c-40cd-9e61-14aab87e1da2 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:34.959530 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:34.964181 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c6900-375c-40cd-9e61-14aab87e1da2 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:35.028704 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ad72fd-3fd1-4ce8-8268-f727a3843621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:35.028704 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad72fd-3fd1-4ce8-8268-f727a3843621 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:35.028704 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad72fd-3fd1-4ce8-8268-f727a3843621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:35.028704 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad72fd-3fd1-4ce8-8268-f727a3843621 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:35.029015 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ad72fd-3fd1-4ce8-8268-f727a3843621 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:35.029799 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ad72fd-3fd1-4ce8-8268-f727a3843621 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:35.053528 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1251/2502] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:26:34 2022] POST /identity/v3/users => generated 303 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:26:35.057936 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fd6a7d-72eb-48df-84b4-d5f933949066 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:35.073458 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fd6a7d-72eb-48df-84b4-d5f933949066 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:35.079064 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464249f1-d0f2-4eaf-87f1-f50957f69880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:35.079736 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464249f1-d0f2-4eaf-87f1-f50957f69880 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:35.080028 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464249f1-d0f2-4eaf-87f1-f50957f69880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:35.080300 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464249f1-d0f2-4eaf-87f1-f50957f69880 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:35.081024 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464249f1-d0f2-4eaf-87f1-f50957f69880 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:35.082647 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464249f1-d0f2-4eaf-87f1-f50957f69880 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:35.096084 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1252/2503] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:26:35.101799 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ad72fd-3fd1-4ce8-8268-f727a3843621 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:35.114615 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ad72fd-3fd1-4ce8-8268-f727a3843621 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:35.119986 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f69726-4e63-4f29-b1c0-c65096ec00b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:35.120883 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f69726-4e63-4f29-b1c0-c65096ec00b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:35.121203 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f69726-4e63-4f29-b1c0-c65096ec00b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:35.121554 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f69726-4e63-4f29-b1c0-c65096ec00b3 admin admin] PATH_INFO: `/v3/projects/bd0d867e64f245179133843ea67ce4b3/users/92a5a8c6da9145e3876d031fb1064a78/roles/f36e053b442540f08041ac7878760f65` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:35.139298 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f69726-4e63-4f29-b1c0-c65096ec00b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd0d867e64f245179133843ea67ce4b3, user_id=92a5a8c6da9145e3876d031fb1064a78, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:35.141976 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f69726-4e63-4f29-b1c0-c65096ec00b3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:35.157154 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1252/2504] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:26:35 2022] PUT /identity/v3/projects/bd0d867e64f245179133843ea67ce4b3/users/92a5a8c6da9145e3876d031fb1064a78/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:26:35.160366 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464249f1-d0f2-4eaf-87f1-f50957f69880 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:35.174713 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464249f1-d0f2-4eaf-87f1-f50957f69880 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:35.257958 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56caca4-5190-4b17-98bc-ad144c038ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:35.258697 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56caca4-5190-4b17-98bc-ad144c038ed0 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:35.258967 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56caca4-5190-4b17-98bc-ad144c038ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:35.259297 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56caca4-5190-4b17-98bc-ad144c038ed0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:35.260067 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56caca4-5190-4b17-98bc-ad144c038ed0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:35.261792 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56caca4-5190-4b17-98bc-ad144c038ed0 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:35.274120 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1253/2505] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:26:35.278200 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f69726-4e63-4f29-b1c0-c65096ec00b3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:35.294111 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f69726-4e63-4f29-b1c0-c65096ec00b3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:35.299039 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6747e201-fd57-4613-8866-e3382caa56eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:35.300425 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6747e201-fd57-4613-8866-e3382caa56eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:35.300425 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6747e201-fd57-4613-8866-e3382caa56eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:35.300613 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6747e201-fd57-4613-8866-e3382caa56eb admin admin] PATH_INFO: `/v3/projects/bd0d867e64f245179133843ea67ce4b3/users/92a5a8c6da9145e3876d031fb1064a78/roles/f36e053b442540f08041ac7878760f65` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:35.302916 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6747e201-fd57-4613-8866-e3382caa56eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd0d867e64f245179133843ea67ce4b3, user_id=92a5a8c6da9145e3876d031fb1064a78, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:35.305520 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6747e201-fd57-4613-8866-e3382caa56eb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:35.319894 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1253/2506] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:26:35 2022] PUT /identity/v3/projects/bd0d867e64f245179133843ea67ce4b3/users/92a5a8c6da9145e3876d031fb1064a78/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:35.331120 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc3e050-13b2-4662-a12c-227af7557c6c None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:35.331423 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc3e050-13b2-4662-a12c-227af7557c6c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:35.331715 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc3e050-13b2-4662-a12c-227af7557c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:35.366342 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6747e201-fd57-4613-8866-e3382caa56eb admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:35.368087 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-fcc3e050-13b2-4662-a12c-227af7557c6c None None] MFA Rules not processed for user `92a5a8c6da9145e3876d031fb1064a78`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:35.368240 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6747e201-fd57-4613-8866-e3382caa56eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:35.369367 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-fcc3e050-13b2-4662-a12c-227af7557c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:35.447157 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6747e201-fd57-4613-8866-e3382caa56eb admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:35.451808 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf9e414-6b77-471a-94a0-656997688bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:35.452682 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf9e414-6b77-471a-94a0-656997688bfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:35.452965 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf9e414-6b77-471a-94a0-656997688bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:35.453272 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf9e414-6b77-471a-94a0-656997688bfe admin admin] PATH_INFO: `/v3/users/0cab15d48dbc47efb36822ccf4aabe84` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:35.454526 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf9e414-6b77-471a-94a0-656997688bfe admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cab15d48dbc47efb36822ccf4aabe84)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:35.456449 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf9e414-6b77-471a-94a0-656997688bfe admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:35.465517 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1254/2507] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:35 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:26:35.476783 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc3e050-13b2-4662-a12c-227af7557c6c None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:35.478268 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-fcc3e050-13b2-4662-a12c-227af7557c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:35.509739 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4bf9e414-6b77-471a-94a0-656997688bfe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cab15d48dbc47efb36822ccf4aabe84'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:35.511583 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4bf9e414-6b77-471a-94a0-656997688bfe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cab15d48dbc47efb36822ccf4aabe84'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:35.518139 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:35.520203 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4bf9e414-6b77-471a-94a0-656997688bfe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cab15d48dbc47efb36822ccf4aabe84'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:35.526504 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=86,keystone:INSERT=6,keystone:DELETE=8 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:35.537855 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4bf9e414-6b77-471a-94a0-656997688bfe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cab15d48dbc47efb36822ccf4aabe84'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:35.538545 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4bf9e414-6b77-471a-94a0-656997688bfe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cab15d48dbc47efb36822ccf4aabe84'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:35.544685 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc3e050-13b2-4662-a12c-227af7557c6c None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:35.548357 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abf2c04-9645-478b-be53-29745b812c89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:35.549000 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abf2c04-9645-478b-be53-29745b812c89 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:35.549313 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abf2c04-9645-478b-be53-29745b812c89 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:35.549592 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abf2c04-9645-478b-be53-29745b812c89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:35.551371 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-1abf2c04-9645-478b-be53-29745b812c89 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:35.552526 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1254/2508] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:26:35 2022] DELETE /identity/v3/users/0cab15d48dbc47efb36822ccf4aabe84 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:26:35.556992 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9253e5bf-2581-4afc-b79a-91cddb5f322d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:35.557404 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9253e5bf-2581-4afc-b79a-91cddb5f322d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:35.557761 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9253e5bf-2581-4afc-b79a-91cddb5f322d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:35.563871 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abf2c04-9645-478b-be53-29745b812c89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:35.565343 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abf2c04-9645-478b-be53-29745b812c89 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:35.576114 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9253e5bf-2581-4afc-b79a-91cddb5f322d None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:35.576388 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1255/2509] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:26:35 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:26:35.577361 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9253e5bf-2581-4afc-b79a-91cddb5f322d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:35.632789 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abf2c04-9645-478b-be53-29745b812c89 service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:35.634215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-1abf2c04-9645-478b-be53-29745b812c89 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:35.639206 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1255/2510] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:26:35.645237 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abf2c04-9645-478b-be53-29745b812c89 service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:35.650042 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9253e5bf-2581-4afc-b79a-91cddb5f322d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:35.651386 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9253e5bf-2581-4afc-b79a-91cddb5f322d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:35.698710 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b1bc75-6bb3-453d-bdfc-9b0c83a182c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:35.699402 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b1bc75-6bb3-453d-bdfc-9b0c83a182c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:35.699686 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b1bc75-6bb3-453d-bdfc-9b0c83a182c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:35.699965 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b1bc75-6bb3-453d-bdfc-9b0c83a182c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:35.701811 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-46b1bc75-6bb3-453d-bdfc-9b0c83a182c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:35.709174 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9253e5bf-2581-4afc-b79a-91cddb5f322d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:35.712846 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6b9e4d-a7bb-40f4-80fe-8ddefeb67a5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:35.713528 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b9e4d-a7bb-40f4-80fe-8ddefeb67a5a service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:35.713819 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b9e4d-a7bb-40f4-80fe-8ddefeb67a5a service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:35.714102 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b9e4d-a7bb-40f4-80fe-8ddefeb67a5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:35.724342 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6b9e4d-a7bb-40f4-80fe-8ddefeb67a5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:35.725780 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6b9e4d-a7bb-40f4-80fe-8ddefeb67a5a service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:35.735436 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1256/2511] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:26:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:26:35.770103 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b1bc75-6bb3-453d-bdfc-9b0c83a182c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:35.771520 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b1bc75-6bb3-453d-bdfc-9b0c83a182c1 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:35.780662 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1256/2512] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:26:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:26:36.025895 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6b9e4d-a7bb-40f4-80fe-8ddefeb67a5a service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:36.027455 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ca6b9e4d-a7bb-40f4-80fe-8ddefeb67a5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:36.042178 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6b9e4d-a7bb-40f4-80fe-8ddefeb67a5a service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:36.066223 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40396a26-d38a-4092-856a-4cf0e08599f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:36.067102 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40396a26-d38a-4092-856a-4cf0e08599f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:36.067388 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40396a26-d38a-4092-856a-4cf0e08599f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:36.067700 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40396a26-d38a-4092-856a-4cf0e08599f5 admin admin] PATH_INFO: `/v3/projects/49bd928841d043708a3f54f4525fcdc7` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:36.069394 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40396a26-d38a-4092-856a-4cf0e08599f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49bd928841d043708a3f54f4525fcdc7)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:36.071295 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40396a26-d38a-4092-856a-4cf0e08599f5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:36.140868 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-40396a26-d38a-4092-856a-4cf0e08599f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49bd928841d043708a3f54f4525fcdc7', 'request_id': 'req-40396a26-d38a-4092-856a-4cf0e08599f5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:36.149097 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-40396a26-d38a-4092-856a-4cf0e08599f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49bd928841d043708a3f54f4525fcdc7', 'request_id': 'req-40396a26-d38a-4092-856a-4cf0e08599f5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:36.158100 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-40396a26-d38a-4092-856a-4cf0e08599f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49bd928841d043708a3f54f4525fcdc7', 'request_id': 'req-40396a26-d38a-4092-856a-4cf0e08599f5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:36.172623 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-40396a26-d38a-4092-856a-4cf0e08599f5 admin admin] The token cache is being invalidate because project 49bd928841d043708a3f54f4525fcdc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:26:36.173116 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-40396a26-d38a-4092-856a-4cf0e08599f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:36.173983 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-40396a26-d38a-4092-856a-4cf0e08599f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:36.176150 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1257/2513] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:26:36 2022] DELETE /identity/v3/projects/49bd928841d043708a3f54f4525fcdc7 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:36.181109 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b1bc75-6bb3-453d-bdfc-9b0c83a182c1 service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:36.193429 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b1bc75-6bb3-453d-bdfc-9b0c83a182c1 service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:36.254752 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0c8b40-ac3c-4b05-b273-81826290073d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:36.255662 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0c8b40-ac3c-4b05-b273-81826290073d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:36.255941 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0c8b40-ac3c-4b05-b273-81826290073d admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:36.256215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0c8b40-ac3c-4b05-b273-81826290073d admin admin] PATH_INFO: `/v3/users/25dc4987d751446c91ffecd953dae7d3` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:36.257462 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0c8b40-ac3c-4b05-b273-81826290073d admin admin] RBAC: Authorizing `identity:delete_user(user_id=25dc4987d751446c91ffecd953dae7d3)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:36.259380 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0c8b40-ac3c-4b05-b273-81826290073d admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:36.293524 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:36.314085 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-3a0c8b40-ac3c-4b05-b273-81826290073d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25dc4987d751446c91ffecd953dae7d3'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:36.323095 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-3a0c8b40-ac3c-4b05-b273-81826290073d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25dc4987d751446c91ffecd953dae7d3'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:36.324303 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-3a0c8b40-ac3c-4b05-b273-81826290073d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25dc4987d751446c91ffecd953dae7d3'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:36.334591 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-3a0c8b40-ac3c-4b05-b273-81826290073d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25dc4987d751446c91ffecd953dae7d3'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:36.351485 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-3a0c8b40-ac3c-4b05-b273-81826290073d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25dc4987d751446c91ffecd953dae7d3'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:36.359314 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1257/2514] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:26:36 2022] DELETE /identity/v3/users/25dc4987d751446c91ffecd953dae7d3 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:36.694307 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40396a26-d38a-4092-856a-4cf0e08599f5 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:36.696629 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-40396a26-d38a-4092-856a-4cf0e08599f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:36.711565 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:36.718473 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40396a26-d38a-4092-856a-4cf0e08599f5 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:36.793539 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5013f43-a394-488e-bfa5-9699801a173b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:36.794408 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5013f43-a394-488e-bfa5-9699801a173b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:36.794692 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5013f43-a394-488e-bfa5-9699801a173b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:36.794971 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5013f43-a394-488e-bfa5-9699801a173b admin admin] PATH_INFO: `/v3/projects/7e8e7c829d814f62bdc33637aba77ae5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:36.796207 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5013f43-a394-488e-bfa5-9699801a173b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e8e7c829d814f62bdc33637aba77ae5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:36.798201 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5013f43-a394-488e-bfa5-9699801a173b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:36.846224 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f5013f43-a394-488e-bfa5-9699801a173b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e8e7c829d814f62bdc33637aba77ae5', 'request_id': 'req-f5013f43-a394-488e-bfa5-9699801a173b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:36.851704 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f5013f43-a394-488e-bfa5-9699801a173b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e8e7c829d814f62bdc33637aba77ae5', 'request_id': 'req-f5013f43-a394-488e-bfa5-9699801a173b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:36.858547 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f5013f43-a394-488e-bfa5-9699801a173b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e8e7c829d814f62bdc33637aba77ae5', 'request_id': 'req-f5013f43-a394-488e-bfa5-9699801a173b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:36.870991 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f5013f43-a394-488e-bfa5-9699801a173b admin admin] The token cache is being invalidate because project 7e8e7c829d814f62bdc33637aba77ae5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:26:36.871284 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f5013f43-a394-488e-bfa5-9699801a173b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:36.871914 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f5013f43-a394-488e-bfa5-9699801a173b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:36.873516 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1258/2515] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:26:36 2022] DELETE /identity/v3/projects/7e8e7c829d814f62bdc33637aba77ae5 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:36.878563 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d88b1e-74c3-4f5f-baa4-59cf26230d7e None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:36.878865 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d88b1e-74c3-4f5f-baa4-59cf26230d7e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:36.879150 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d88b1e-74c3-4f5f-baa4-59cf26230d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:36.906100 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-31d88b1e-74c3-4f5f-baa4-59cf26230d7e None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:36.908263 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-31d88b1e-74c3-4f5f-baa4-59cf26230d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:36.971424 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1258/2516] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:26:36.979445 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b691164b-dfec-49fa-be27-e1fdb6f39971 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:36.979738 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b691164b-dfec-49fa-be27-e1fdb6f39971 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:36.980021 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b691164b-dfec-49fa-be27-e1fdb6f39971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:36.998723 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b691164b-dfec-49fa-be27-e1fdb6f39971 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:36.999939 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b691164b-dfec-49fa-be27-e1fdb6f39971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:37.008364 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1259/2517] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:26:37.012542 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d88b1e-74c3-4f5f-baa4-59cf26230d7e None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:37.023000 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d88b1e-74c3-4f5f-baa4-59cf26230d7e None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:37.027591 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c79708-3aed-463d-815f-fc098bb19ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:37.028269 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c79708-3aed-463d-815f-fc098bb19ed2 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:37.028577 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c79708-3aed-463d-815f-fc098bb19ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:37.028851 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c79708-3aed-463d-815f-fc098bb19ed2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:37.029605 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c79708-3aed-463d-815f-fc098bb19ed2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:26:37.029910 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c79708-3aed-463d-815f-fc098bb19ed2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:37.031508 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c79708-3aed-463d-815f-fc098bb19ed2 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:37.042909 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1259/2518] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:26:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:26:37.047260 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b691164b-dfec-49fa-be27-e1fdb6f39971 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:37.056802 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b691164b-dfec-49fa-be27-e1fdb6f39971 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:37.061463 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47d228f-292b-4ff0-942d-dcf2239e693a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:37.062107 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d228f-292b-4ff0-942d-dcf2239e693a admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:37.062392 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d228f-292b-4ff0-942d-dcf2239e693a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:37.062675 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d228f-292b-4ff0-942d-dcf2239e693a admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:37.063379 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47d228f-292b-4ff0-942d-dcf2239e693a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:37.065333 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47d228f-292b-4ff0-942d-dcf2239e693a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:37.096958 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1260/2519] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:26:37 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:26:37.101732 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c79708-3aed-463d-815f-fc098bb19ed2 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:37.111564 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c79708-3aed-463d-815f-fc098bb19ed2 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:37.173351 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ef2d2c-ebce-4af9-bc86-e35d7aa23a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:37.174168 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef2d2c-ebce-4af9-bc86-e35d7aa23a5f admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:37.174456 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef2d2c-ebce-4af9-bc86-e35d7aa23a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:37.174733 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef2d2c-ebce-4af9-bc86-e35d7aa23a5f admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:37.175418 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ef2d2c-ebce-4af9-bc86-e35d7aa23a5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:37.177879 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ef2d2c-ebce-4af9-bc86-e35d7aa23a5f admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:37.203859 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1260/2520] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:26:37 2022] POST /identity/v3/users => generated 286 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:26:37.205289 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47d228f-292b-4ff0-942d-dcf2239e693a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:37.217224 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47d228f-292b-4ff0-942d-dcf2239e693a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:37.222750 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e265f6ee-058c-45b3-8623-95fb652720db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:37.223415 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e265f6ee-058c-45b3-8623-95fb652720db admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:37.223723 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e265f6ee-058c-45b3-8623-95fb652720db admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:37.224255 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e265f6ee-058c-45b3-8623-95fb652720db admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:37.225022 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e265f6ee-058c-45b3-8623-95fb652720db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:37.226792 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e265f6ee-058c-45b3-8623-95fb652720db admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:37.242202 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1261/2521] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:26:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:26:37.243609 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ef2d2c-ebce-4af9-bc86-e35d7aa23a5f admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:37.253676 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ef2d2c-ebce-4af9-bc86-e35d7aa23a5f admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:37.259064 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a668ad4e-b4f4-4636-bca3-fc16d2d132ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:37.259719 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668ad4e-b4f4-4636-bca3-fc16d2d132ac admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:37.260111 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668ad4e-b4f4-4636-bca3-fc16d2d132ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:37.260267 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668ad4e-b4f4-4636-bca3-fc16d2d132ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:37.261395 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a668ad4e-b4f4-4636-bca3-fc16d2d132ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:37.263074 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a668ad4e-b4f4-4636-bca3-fc16d2d132ac admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:37.276386 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1261/2522] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:26:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:26:37.277871 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e265f6ee-058c-45b3-8623-95fb652720db admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:37.288146 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e265f6ee-058c-45b3-8623-95fb652720db admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:37.292882 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fc1135-8d2c-47f3-b78c-5f1b4b5e8319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:37.293784 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fc1135-8d2c-47f3-b78c-5f1b4b5e8319 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:37.294072 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fc1135-8d2c-47f3-b78c-5f1b4b5e8319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:37.294352 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fc1135-8d2c-47f3-b78c-5f1b4b5e8319 admin admin] PATH_INFO: `/v3/projects/21835652c80b4c719b04bb090eba6cd0/users/9390ae25a8294693868b892ec92496a0/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:37.311021 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fc1135-8d2c-47f3-b78c-5f1b4b5e8319 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21835652c80b4c719b04bb090eba6cd0, user_id=9390ae25a8294693868b892ec92496a0, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:37.313743 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fc1135-8d2c-47f3-b78c-5f1b4b5e8319 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:37.328405 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1262/2523] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:26:37 2022] PUT /identity/v3/projects/21835652c80b4c719b04bb090eba6cd0/users/9390ae25a8294693868b892ec92496a0/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:37.339262 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1202d44d-f51a-4b37-8c2e-a8a65eed3f31 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:37.339753 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1202d44d-f51a-4b37-8c2e-a8a65eed3f31 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:37.340065 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1202d44d-f51a-4b37-8c2e-a8a65eed3f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:37.364865 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:37.370555 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-1202d44d-f51a-4b37-8c2e-a8a65eed3f31 None None] MFA Rules not processed for user `9390ae25a8294693868b892ec92496a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:37.372604 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-1202d44d-f51a-4b37-8c2e-a8a65eed3f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:37.441079 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1262/2524] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:26:37.453803 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fc1135-8d2c-47f3-b78c-5f1b4b5e8319 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:37.455689 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b2fc1135-8d2c-47f3-b78c-5f1b4b5e8319 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:37.492390 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:37.528338 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fc1135-8d2c-47f3-b78c-5f1b4b5e8319 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:37.532273 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca41dc-7f1f-40fe-896e-f49d35ab26c6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '192724527d774817bc4581474ac848dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:37.532931 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca41dc-7f1f-40fe-896e-f49d35ab26c6 service glance] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:37.533259 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca41dc-7f1f-40fe-896e-f49d35ab26c6 service glance] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:37.533537 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca41dc-7f1f-40fe-896e-f49d35ab26c6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:37.543454 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ca41dc-7f1f-40fe-896e-f49d35ab26c6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:37.544883 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ca41dc-7f1f-40fe-896e-f49d35ab26c6 service glance] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:37.555174 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1263/2525] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:26:37 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:26:40.350792 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1202d44d-f51a-4b37-8c2e-a8a65eed3f31 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:40.352281 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-1202d44d-f51a-4b37-8c2e-a8a65eed3f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:40.420086 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1202d44d-f51a-4b37-8c2e-a8a65eed3f31 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:40.424556 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f2dbd6-b9ac-4a0f-b6c5-9ac78494c210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:40.425481 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f2dbd6-b9ac-4a0f-b6c5-9ac78494c210 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:40.425796 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f2dbd6-b9ac-4a0f-b6c5-9ac78494c210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:40.426075 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f2dbd6-b9ac-4a0f-b6c5-9ac78494c210 admin admin] PATH_INFO: `/v3/users/8b66d6b276964415b846d46ec5363782` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:40.427625 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f2dbd6-b9ac-4a0f-b6c5-9ac78494c210 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b66d6b276964415b846d46ec5363782)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:40.429579 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f2dbd6-b9ac-4a0f-b6c5-9ac78494c210 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:40.477275 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-84f2dbd6-b9ac-4a0f-b6c5-9ac78494c210 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b66d6b276964415b846d46ec5363782'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:40.484509 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-84f2dbd6-b9ac-4a0f-b6c5-9ac78494c210 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b66d6b276964415b846d46ec5363782'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:40.485301 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-84f2dbd6-b9ac-4a0f-b6c5-9ac78494c210 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b66d6b276964415b846d46ec5363782'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:40.492482 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-84f2dbd6-b9ac-4a0f-b6c5-9ac78494c210 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b66d6b276964415b846d46ec5363782'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:40.505029 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-84f2dbd6-b9ac-4a0f-b6c5-9ac78494c210 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b66d6b276964415b846d46ec5363782'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:40.511067 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1263/2526] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:26:40 2022] DELETE /identity/v3/users/8b66d6b276964415b846d46ec5363782 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:40.515522 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77dc37-e1fd-4c09-9aa3-84c00ea39655 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:40.516034 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77dc37-e1fd-4c09-9aa3-84c00ea39655 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:40.516134 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77dc37-e1fd-4c09-9aa3-84c00ea39655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:40.533620 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4f77dc37-e1fd-4c09-9aa3-84c00ea39655 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:40.534832 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4f77dc37-e1fd-4c09-9aa3-84c00ea39655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:40.593973 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1264/2527] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:26:40.603763 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f2dbd6-b9ac-4a0f-b6c5-9ac78494c210 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:40.605238 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-84f2dbd6-b9ac-4a0f-b6c5-9ac78494c210 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:40.632044 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:40.664967 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f2dbd6-b9ac-4a0f-b6c5-9ac78494c210 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:40.668684 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b8df2-9305-4e31-a7c8-b17b947c4a5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:40.669372 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b8df2-9305-4e31-a7c8-b17b947c4a5d service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:40.669663 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b8df2-9305-4e31-a7c8-b17b947c4a5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:40.669949 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b8df2-9305-4e31-a7c8-b17b947c4a5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:40.679766 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145b8df2-9305-4e31-a7c8-b17b947c4a5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:40.681441 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145b8df2-9305-4e31-a7c8-b17b947c4a5d service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:40.691849 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1264/2528] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:26:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:26:41.003093 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f77dc37-e1fd-4c09-9aa3-84c00ea39655 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:41.004931 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4f77dc37-e1fd-4c09-9aa3-84c00ea39655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:41.022297 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f77dc37-e1fd-4c09-9aa3-84c00ea39655 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:41.030683 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd32c0-7aff-40e0-a765-4734b213f834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:41.032300 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd32c0-7aff-40e0-a765-4734b213f834 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:41.032912 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd32c0-7aff-40e0-a765-4734b213f834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:41.033571 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd32c0-7aff-40e0-a765-4734b213f834 admin admin] PATH_INFO: `/v3/projects/cf1fdcca16e84fe3907ea25eb02957ec` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:41.035829 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbd32c0-7aff-40e0-a765-4734b213f834 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf1fdcca16e84fe3907ea25eb02957ec)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:41.039096 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbd32c0-7aff-40e0-a765-4734b213f834 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:41.091463 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cfbd32c0-7aff-40e0-a765-4734b213f834 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf1fdcca16e84fe3907ea25eb02957ec', 'request_id': 'req-cfbd32c0-7aff-40e0-a765-4734b213f834'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:41.097292 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cfbd32c0-7aff-40e0-a765-4734b213f834 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf1fdcca16e84fe3907ea25eb02957ec', 'request_id': 'req-cfbd32c0-7aff-40e0-a765-4734b213f834'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:41.104822 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cfbd32c0-7aff-40e0-a765-4734b213f834 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf1fdcca16e84fe3907ea25eb02957ec', 'request_id': 'req-cfbd32c0-7aff-40e0-a765-4734b213f834'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:41.118182 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cfbd32c0-7aff-40e0-a765-4734b213f834 admin admin] The token cache is being invalidate because project cf1fdcca16e84fe3907ea25eb02957ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:26:41.118696 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cfbd32c0-7aff-40e0-a765-4734b213f834 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:41.119302 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cfbd32c0-7aff-40e0-a765-4734b213f834 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:41.121086 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1265/2529] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:26:41 2022] DELETE /identity/v3/projects/cf1fdcca16e84fe3907ea25eb02957ec => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:26:41.125282 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b8df2-9305-4e31-a7c8-b17b947c4a5d service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:41.136985 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b8df2-9305-4e31-a7c8-b17b947c4a5d service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:41.208847 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc70cb53-0dfa-429c-95ff-87df3c69fab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:41.209919 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70cb53-0dfa-429c-95ff-87df3c69fab2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:41.210182 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70cb53-0dfa-429c-95ff-87df3c69fab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:41.210620 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70cb53-0dfa-429c-95ff-87df3c69fab2 admin admin] PATH_INFO: `/v3/users/756dff54b082436ca12786797a0b94d8` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:41.212106 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc70cb53-0dfa-429c-95ff-87df3c69fab2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=756dff54b082436ca12786797a0b94d8)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:41.214506 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc70cb53-0dfa-429c-95ff-87df3c69fab2 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:41.269003 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-bc70cb53-0dfa-429c-95ff-87df3c69fab2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '756dff54b082436ca12786797a0b94d8'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:41.279974 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-bc70cb53-0dfa-429c-95ff-87df3c69fab2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '756dff54b082436ca12786797a0b94d8'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:41.280905 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-bc70cb53-0dfa-429c-95ff-87df3c69fab2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '756dff54b082436ca12786797a0b94d8'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:41.289629 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-bc70cb53-0dfa-429c-95ff-87df3c69fab2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '756dff54b082436ca12786797a0b94d8'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:41.310092 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-bc70cb53-0dfa-429c-95ff-87df3c69fab2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '756dff54b082436ca12786797a0b94d8'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:41.317168 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1265/2530] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:26:41 2022] DELETE /identity/v3/users/756dff54b082436ca12786797a0b94d8 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:41.608675 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd32c0-7aff-40e0-a765-4734b213f834 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:41.610159 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cfbd32c0-7aff-40e0-a765-4734b213f834 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:41.624709 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd32c0-7aff-40e0-a765-4734b213f834 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:41.678398 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:41.686777 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4550c592-c464-4bbf-9dd9-b6ca50988bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:41.687866 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4550c592-c464-4bbf-9dd9-b6ca50988bca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:41.688368 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4550c592-c464-4bbf-9dd9-b6ca50988bca admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:41.688857 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4550c592-c464-4bbf-9dd9-b6ca50988bca admin admin] PATH_INFO: `/v3/projects/8bef283e7e7e4e76b3c81e1e1cb6c060` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:41.690322 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4550c592-c464-4bbf-9dd9-b6ca50988bca admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bef283e7e7e4e76b3c81e1e1cb6c060)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:41.692331 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4550c592-c464-4bbf-9dd9-b6ca50988bca admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:41.737905 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4550c592-c464-4bbf-9dd9-b6ca50988bca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bef283e7e7e4e76b3c81e1e1cb6c060', 'request_id': 'req-4550c592-c464-4bbf-9dd9-b6ca50988bca'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:41.743954 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4550c592-c464-4bbf-9dd9-b6ca50988bca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bef283e7e7e4e76b3c81e1e1cb6c060', 'request_id': 'req-4550c592-c464-4bbf-9dd9-b6ca50988bca'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:41.750121 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4550c592-c464-4bbf-9dd9-b6ca50988bca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bef283e7e7e4e76b3c81e1e1cb6c060', 'request_id': 'req-4550c592-c464-4bbf-9dd9-b6ca50988bca'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:41.762540 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4550c592-c464-4bbf-9dd9-b6ca50988bca admin admin] The token cache is being invalidate because project 8bef283e7e7e4e76b3c81e1e1cb6c060 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:26:41.762832 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4550c592-c464-4bbf-9dd9-b6ca50988bca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:41.763446 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4550c592-c464-4bbf-9dd9-b6ca50988bca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:41.764979 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1266/2531] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:26:41 2022] DELETE /identity/v3/projects/8bef283e7e7e4e76b3c81e1e1cb6c060 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:41.769967 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b7a6b0-286b-49a6-b138-584bb1dc00d8 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:41.770472 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b7a6b0-286b-49a6-b138-584bb1dc00d8 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:41.771299 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b7a6b0-286b-49a6-b138-584bb1dc00d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:41.796137 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-54b7a6b0-286b-49a6-b138-584bb1dc00d8 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:41.797454 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-54b7a6b0-286b-49a6-b138-584bb1dc00d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:41.838594 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:41.859798 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1266/2532] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:26:41.867917 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986f779-f917-471d-8bcb-0ce1a149058f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:41.868209 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986f779-f917-471d-8bcb-0ce1a149058f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:41.868489 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986f779-f917-471d-8bcb-0ce1a149058f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:41.886840 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3986f779-f917-471d-8bcb-0ce1a149058f None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:41.888031 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3986f779-f917-471d-8bcb-0ce1a149058f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:41.896809 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1267/2533] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:26:41.900833 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b7a6b0-286b-49a6-b138-584bb1dc00d8 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:41.910359 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b7a6b0-286b-49a6-b138-584bb1dc00d8 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:41.915969 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76a4953-eba0-40c7-8fc0-25df8822a8ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:41.916616 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a4953-eba0-40c7-8fc0-25df8822a8ce admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:41.916903 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a4953-eba0-40c7-8fc0-25df8822a8ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:41.917223 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a4953-eba0-40c7-8fc0-25df8822a8ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:41.917944 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76a4953-eba0-40c7-8fc0-25df8822a8ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:26:41.918251 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76a4953-eba0-40c7-8fc0-25df8822a8ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:41.919834 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76a4953-eba0-40c7-8fc0-25df8822a8ce admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:41.931555 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1267/2534] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:26:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:26:41.935804 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3986f779-f917-471d-8bcb-0ce1a149058f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:41.945194 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3986f779-f917-471d-8bcb-0ce1a149058f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:41.949817 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff61ade-de8a-4f44-bee5-0bf5dbb1e511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:41.950469 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff61ade-de8a-4f44-bee5-0bf5dbb1e511 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:41.950758 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff61ade-de8a-4f44-bee5-0bf5dbb1e511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:41.951035 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff61ade-de8a-4f44-bee5-0bf5dbb1e511 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:41.951726 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff61ade-de8a-4f44-bee5-0bf5dbb1e511 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:41.953651 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff61ade-de8a-4f44-bee5-0bf5dbb1e511 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:41.980628 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1268/2535] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:26:41 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:26:41.984915 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76a4953-eba0-40c7-8fc0-25df8822a8ce admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:41.995024 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76a4953-eba0-40c7-8fc0-25df8822a8ce admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:42.052172 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782034c7-12f3-4772-bb6d-ec524b9eb2f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:42.052842 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782034c7-12f3-4772-bb6d-ec524b9eb2f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:42.053162 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782034c7-12f3-4772-bb6d-ec524b9eb2f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:42.053444 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782034c7-12f3-4772-bb6d-ec524b9eb2f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:42.054141 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782034c7-12f3-4772-bb6d-ec524b9eb2f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:42.056056 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782034c7-12f3-4772-bb6d-ec524b9eb2f2 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:42.076745 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1268/2536] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:26:41 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:26:42.080844 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff61ade-de8a-4f44-bee5-0bf5dbb1e511 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:42.092230 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff61ade-de8a-4f44-bee5-0bf5dbb1e511 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:42.097165 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a033d4-300d-4ed5-954f-4e6a53caf83a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:42.097821 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a033d4-300d-4ed5-954f-4e6a53caf83a admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:42.098101 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a033d4-300d-4ed5-954f-4e6a53caf83a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:42.098377 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a033d4-300d-4ed5-954f-4e6a53caf83a admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:42.099543 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a033d4-300d-4ed5-954f-4e6a53caf83a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:42.101145 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a033d4-300d-4ed5-954f-4e6a53caf83a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:42.111345 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1269/2537] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:26:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:26:42.115318 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782034c7-12f3-4772-bb6d-ec524b9eb2f2 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:42.125402 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782034c7-12f3-4772-bb6d-ec524b9eb2f2 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:42.130592 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6c02f8-6a79-45d9-9c2e-c9b5bb78a3d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:42.131341 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c02f8-6a79-45d9-9c2e-c9b5bb78a3d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:42.131633 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c02f8-6a79-45d9-9c2e-c9b5bb78a3d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:42.131924 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c02f8-6a79-45d9-9c2e-c9b5bb78a3d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:42.132651 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6c02f8-6a79-45d9-9c2e-c9b5bb78a3d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:42.134298 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6c02f8-6a79-45d9-9c2e-c9b5bb78a3d6 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:42.144209 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1269/2538] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:26:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:26:42.148869 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a033d4-300d-4ed5-954f-4e6a53caf83a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:42.158677 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a033d4-300d-4ed5-954f-4e6a53caf83a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:42.163297 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df72d567-38c7-49df-96e2-e01623938b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:42.164159 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df72d567-38c7-49df-96e2-e01623938b1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:42.164444 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df72d567-38c7-49df-96e2-e01623938b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:42.164731 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df72d567-38c7-49df-96e2-e01623938b1b admin admin] PATH_INFO: `/v3/projects/60a3a514b11b4c3eb02ba537fb75d577/users/bfd08eca01724dd2b8ea0d4196692b80/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:42.179302 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df72d567-38c7-49df-96e2-e01623938b1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=60a3a514b11b4c3eb02ba537fb75d577, user_id=bfd08eca01724dd2b8ea0d4196692b80, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:42.182022 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df72d567-38c7-49df-96e2-e01623938b1b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:42.196052 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1270/2539] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:26:42 2022] PUT /identity/v3/projects/60a3a514b11b4c3eb02ba537fb75d577/users/bfd08eca01724dd2b8ea0d4196692b80/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:42.208869 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e3c949-d578-4c3e-a889-226aace58400 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:42.209503 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e3c949-d578-4c3e-a889-226aace58400 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:42.209959 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e3c949-d578-4c3e-a889-226aace58400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:42.237215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-c6e3c949-d578-4c3e-a889-226aace58400 None None] MFA Rules not processed for user `bfd08eca01724dd2b8ea0d4196692b80`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:42.238433 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c6e3c949-d578-4c3e-a889-226aace58400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:42.301865 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1270/2540] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:26:42.312504 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df72d567-38c7-49df-96e2-e01623938b1b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:42.314252 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-df72d567-38c7-49df-96e2-e01623938b1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:42.373361 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df72d567-38c7-49df-96e2-e01623938b1b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:42.377028 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35df4e65-b07f-4c7f-9470-18faeec95230 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:42.377708 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df4e65-b07f-4c7f-9470-18faeec95230 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:42.377995 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df4e65-b07f-4c7f-9470-18faeec95230 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:42.378278 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df4e65-b07f-4c7f-9470-18faeec95230 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:42.397543 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35df4e65-b07f-4c7f-9470-18faeec95230 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:42.399204 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35df4e65-b07f-4c7f-9470-18faeec95230 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:42.409500 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1271/2541] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:26:42 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:26:43.732537 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e3c949-d578-4c3e-a889-226aace58400 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:43.734131 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c6e3c949-d578-4c3e-a889-226aace58400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:43.779158 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:43.811719 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e3c949-d578-4c3e-a889-226aace58400 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:43.811719 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68435b61-ac69-4331-bb91-2d3969e8988e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:43.811719 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68435b61-ac69-4331-bb91-2d3969e8988e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:43.811719 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68435b61-ac69-4331-bb91-2d3969e8988e admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:43.812283 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68435b61-ac69-4331-bb91-2d3969e8988e admin admin] PATH_INFO: `/v3/users/e17fd977523f4265b5fd472f8ba2272c` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:43.812658 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68435b61-ac69-4331-bb91-2d3969e8988e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e17fd977523f4265b5fd472f8ba2272c)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:43.814999 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68435b61-ac69-4331-bb91-2d3969e8988e admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:43.878138 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-68435b61-ac69-4331-bb91-2d3969e8988e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e17fd977523f4265b5fd472f8ba2272c'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:43.885926 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-68435b61-ac69-4331-bb91-2d3969e8988e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e17fd977523f4265b5fd472f8ba2272c'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:43.886557 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-68435b61-ac69-4331-bb91-2d3969e8988e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e17fd977523f4265b5fd472f8ba2272c'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:43.894246 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-68435b61-ac69-4331-bb91-2d3969e8988e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e17fd977523f4265b5fd472f8ba2272c'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:43.914362 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-68435b61-ac69-4331-bb91-2d3969e8988e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e17fd977523f4265b5fd472f8ba2272c'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:43.920650 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1271/2542] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:26:43 2022] DELETE /identity/v3/users/e17fd977523f4265b5fd472f8ba2272c => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:43.925317 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc655d5-a479-4838-b3ac-f9382fdede8c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:43.925840 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc655d5-a479-4838-b3ac-f9382fdede8c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:43.926007 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc655d5-a479-4838-b3ac-f9382fdede8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:43.945141 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9fc655d5-a479-4838-b3ac-f9382fdede8c None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:43.946382 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9fc655d5-a479-4838-b3ac-f9382fdede8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:43.966515 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:44.015621 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1272/2543] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:26:44.026349 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68435b61-ac69-4331-bb91-2d3969e8988e admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:44.027795 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-68435b61-ac69-4331-bb91-2d3969e8988e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:44.108627 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68435b61-ac69-4331-bb91-2d3969e8988e admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:44.113279 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990cfbad-6096-4100-9bab-909f511ffba9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:44.114595 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990cfbad-6096-4100-9bab-909f511ffba9 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:44.115164 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990cfbad-6096-4100-9bab-909f511ffba9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:44.116488 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990cfbad-6096-4100-9bab-909f511ffba9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:44.127193 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990cfbad-6096-4100-9bab-909f511ffba9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:44.129231 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990cfbad-6096-4100-9bab-909f511ffba9 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:44.141933 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1272/2544] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:26:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:26:44.441311 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc655d5-a479-4838-b3ac-f9382fdede8c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:44.443247 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9fc655d5-a479-4838-b3ac-f9382fdede8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:44.456550 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc655d5-a479-4838-b3ac-f9382fdede8c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:44.466552 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c16eac0-32df-4ecc-80da-3268281de179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:44.467461 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c16eac0-32df-4ecc-80da-3268281de179 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:44.467754 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c16eac0-32df-4ecc-80da-3268281de179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:44.468037 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c16eac0-32df-4ecc-80da-3268281de179 admin admin] PATH_INFO: `/v3/projects/c60fbde647bb4e7c9e043fc44bc4cab8` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:44.469605 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c16eac0-32df-4ecc-80da-3268281de179 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c60fbde647bb4e7c9e043fc44bc4cab8)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:44.471516 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c16eac0-32df-4ecc-80da-3268281de179 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:44.532838 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1c16eac0-32df-4ecc-80da-3268281de179 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c60fbde647bb4e7c9e043fc44bc4cab8', 'request_id': 'req-1c16eac0-32df-4ecc-80da-3268281de179'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:44.542046 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1c16eac0-32df-4ecc-80da-3268281de179 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c60fbde647bb4e7c9e043fc44bc4cab8', 'request_id': 'req-1c16eac0-32df-4ecc-80da-3268281de179'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:44.549312 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1c16eac0-32df-4ecc-80da-3268281de179 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c60fbde647bb4e7c9e043fc44bc4cab8', 'request_id': 'req-1c16eac0-32df-4ecc-80da-3268281de179'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:44.564441 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1c16eac0-32df-4ecc-80da-3268281de179 admin admin] The token cache is being invalidate because project c60fbde647bb4e7c9e043fc44bc4cab8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:26:44.564985 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1c16eac0-32df-4ecc-80da-3268281de179 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:44.565780 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1c16eac0-32df-4ecc-80da-3268281de179 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:44.568163 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1273/2545] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:26:44 2022] DELETE /identity/v3/projects/c60fbde647bb4e7c9e043fc44bc4cab8 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:44.572125 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990cfbad-6096-4100-9bab-909f511ffba9 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:44.582654 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990cfbad-6096-4100-9bab-909f511ffba9 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:44.649478 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fb5b8c-ddd9-45e6-8ce6-7d6d3b9eacd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:44.650503 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fb5b8c-ddd9-45e6-8ce6-7d6d3b9eacd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:44.650844 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fb5b8c-ddd9-45e6-8ce6-7d6d3b9eacd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:44.651158 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fb5b8c-ddd9-45e6-8ce6-7d6d3b9eacd7 admin admin] PATH_INFO: `/v3/users/92a5a8c6da9145e3876d031fb1064a78` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:44.652514 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fb5b8c-ddd9-45e6-8ce6-7d6d3b9eacd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92a5a8c6da9145e3876d031fb1064a78)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:44.654801 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fb5b8c-ddd9-45e6-8ce6-7d6d3b9eacd7 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:44.658233 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:44.713314 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a0fb5b8c-ddd9-45e6-8ce6-7d6d3b9eacd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92a5a8c6da9145e3876d031fb1064a78'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:44.721974 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a0fb5b8c-ddd9-45e6-8ce6-7d6d3b9eacd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92a5a8c6da9145e3876d031fb1064a78'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:44.722645 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a0fb5b8c-ddd9-45e6-8ce6-7d6d3b9eacd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92a5a8c6da9145e3876d031fb1064a78'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:44.730723 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a0fb5b8c-ddd9-45e6-8ce6-7d6d3b9eacd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92a5a8c6da9145e3876d031fb1064a78'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:44.744127 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a0fb5b8c-ddd9-45e6-8ce6-7d6d3b9eacd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92a5a8c6da9145e3876d031fb1064a78'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:44.750894 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1273/2546] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:26:44 2022] DELETE /identity/v3/users/92a5a8c6da9145e3876d031fb1064a78 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:45.040459 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c16eac0-32df-4ecc-80da-3268281de179 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:45.042182 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1c16eac0-32df-4ecc-80da-3268281de179 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:45.056449 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c16eac0-32df-4ecc-80da-3268281de179 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:45.119896 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6affb1d9-c03d-4eaa-9824-42e6b057f915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:45.120776 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6affb1d9-c03d-4eaa-9824-42e6b057f915 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:45.121207 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6affb1d9-c03d-4eaa-9824-42e6b057f915 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:45.121491 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6affb1d9-c03d-4eaa-9824-42e6b057f915 admin admin] PATH_INFO: `/v3/projects/bd0d867e64f245179133843ea67ce4b3` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:45.122788 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6affb1d9-c03d-4eaa-9824-42e6b057f915 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd0d867e64f245179133843ea67ce4b3)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:45.125216 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6affb1d9-c03d-4eaa-9824-42e6b057f915 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:45.185930 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6affb1d9-c03d-4eaa-9824-42e6b057f915 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd0d867e64f245179133843ea67ce4b3', 'request_id': 'req-6affb1d9-c03d-4eaa-9824-42e6b057f915'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:45.187154 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:45.191832 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6affb1d9-c03d-4eaa-9824-42e6b057f915 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd0d867e64f245179133843ea67ce4b3', 'request_id': 'req-6affb1d9-c03d-4eaa-9824-42e6b057f915'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:45.200742 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6affb1d9-c03d-4eaa-9824-42e6b057f915 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd0d867e64f245179133843ea67ce4b3', 'request_id': 'req-6affb1d9-c03d-4eaa-9824-42e6b057f915'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:45.215832 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6affb1d9-c03d-4eaa-9824-42e6b057f915 admin admin] The token cache is being invalidate because project bd0d867e64f245179133843ea67ce4b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:26:45.216128 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6affb1d9-c03d-4eaa-9824-42e6b057f915 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:45.216765 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6affb1d9-c03d-4eaa-9824-42e6b057f915 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:45.219118 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1274/2547] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:26:45 2022] DELETE /identity/v3/projects/bd0d867e64f245179133843ea67ce4b3 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:45.225015 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf399aa9-d53c-4224-a23f-eebdc1ea9012 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:45.225536 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf399aa9-d53c-4224-a23f-eebdc1ea9012 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:45.225674 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf399aa9-d53c-4224-a23f-eebdc1ea9012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:45.246247 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-cf399aa9-d53c-4224-a23f-eebdc1ea9012 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:45.247380 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-cf399aa9-d53c-4224-a23f-eebdc1ea9012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:45.313457 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1274/2548] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:26:45.321707 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e251fe-77f8-4de4-b5ac-fcf66c0fadeb None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:45.322029 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e251fe-77f8-4de4-b5ac-fcf66c0fadeb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:45.322346 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e251fe-77f8-4de4-b5ac-fcf66c0fadeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:45.341529 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d6e251fe-77f8-4de4-b5ac-fcf66c0fadeb None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:45.342756 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d6e251fe-77f8-4de4-b5ac-fcf66c0fadeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:45.355479 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1275/2549] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:26:45.356836 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf399aa9-d53c-4224-a23f-eebdc1ea9012 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:45.372367 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf399aa9-d53c-4224-a23f-eebdc1ea9012 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:45.377205 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2285387a-5c03-4f65-8a38-0881b1f28d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:45.377880 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2285387a-5c03-4f65-8a38-0881b1f28d40 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:45.378167 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2285387a-5c03-4f65-8a38-0881b1f28d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:45.378444 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2285387a-5c03-4f65-8a38-0881b1f28d40 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:45.379171 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2285387a-5c03-4f65-8a38-0881b1f28d40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:26:45.379493 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2285387a-5c03-4f65-8a38-0881b1f28d40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:45.381462 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2285387a-5c03-4f65-8a38-0881b1f28d40 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:45.395157 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1275/2550] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:26:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:26:45.399652 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e251fe-77f8-4de4-b5ac-fcf66c0fadeb None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:45.410611 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e251fe-77f8-4de4-b5ac-fcf66c0fadeb None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:45.418446 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efdb29c-f228-4b79-94fd-938c4b026885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:45.419251 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efdb29c-f228-4b79-94fd-938c4b026885 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:45.419579 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efdb29c-f228-4b79-94fd-938c4b026885 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:45.419894 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efdb29c-f228-4b79-94fd-938c4b026885 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:45.420606 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efdb29c-f228-4b79-94fd-938c4b026885 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:45.422567 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efdb29c-f228-4b79-94fd-938c4b026885 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:45.452007 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1276/2551] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:26:45 2022] POST /identity/v3/projects => generated 389 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:26:45.456278 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2285387a-5c03-4f65-8a38-0881b1f28d40 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:45.468295 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2285387a-5c03-4f65-8a38-0881b1f28d40 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:45.545544 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2932024-0514-48ec-b7a3-aedce59891cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:45.546226 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2932024-0514-48ec-b7a3-aedce59891cc admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:45.546515 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2932024-0514-48ec-b7a3-aedce59891cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:45.546799 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2932024-0514-48ec-b7a3-aedce59891cc admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:45.547500 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2932024-0514-48ec-b7a3-aedce59891cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:45.549504 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2932024-0514-48ec-b7a3-aedce59891cc admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:45.572820 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:45.591043 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1276/2552] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:26:45 2022] POST /identity/v3/users => generated 295 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:26:45.594339 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efdb29c-f228-4b79-94fd-938c4b026885 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:45.608220 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efdb29c-f228-4b79-94fd-938c4b026885 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:45.612154 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add779b6-f292-4b6d-9fef-60af229241f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:45.612911 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add779b6-f292-4b6d-9fef-60af229241f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:45.613254 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add779b6-f292-4b6d-9fef-60af229241f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:45.613605 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add779b6-f292-4b6d-9fef-60af229241f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:45.614569 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add779b6-f292-4b6d-9fef-60af229241f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:45.616329 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add779b6-f292-4b6d-9fef-60af229241f3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:45.629278 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1277/2553] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:26:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:26:45.633329 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2932024-0514-48ec-b7a3-aedce59891cc admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:45.643215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2932024-0514-48ec-b7a3-aedce59891cc admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:45.649380 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97c0540-bf1e-4735-afca-41cbfffa44fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:45.650082 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97c0540-bf1e-4735-afca-41cbfffa44fb admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:45.650399 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97c0540-bf1e-4735-afca-41cbfffa44fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:45.650684 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97c0540-bf1e-4735-afca-41cbfffa44fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:45.651410 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97c0540-bf1e-4735-afca-41cbfffa44fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:45.653233 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97c0540-bf1e-4735-afca-41cbfffa44fb admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:45.663995 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1277/2554] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:26:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:26:45.668036 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add779b6-f292-4b6d-9fef-60af229241f3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:45.678110 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add779b6-f292-4b6d-9fef-60af229241f3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:45.682903 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672dbed0-17ed-47a4-94ea-8a5f50d6af56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:45.683779 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672dbed0-17ed-47a4-94ea-8a5f50d6af56 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:45.684068 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672dbed0-17ed-47a4-94ea-8a5f50d6af56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:45.684347 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672dbed0-17ed-47a4-94ea-8a5f50d6af56 admin admin] PATH_INFO: `/v3/projects/7b7def58339d4553a08c29ab7683a630/users/68db9045a1414a208b0e69ddab7671e4/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:45.698824 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672dbed0-17ed-47a4-94ea-8a5f50d6af56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b7def58339d4553a08c29ab7683a630, user_id=68db9045a1414a208b0e69ddab7671e4, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:45.701533 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672dbed0-17ed-47a4-94ea-8a5f50d6af56 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:45.714906 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1278/2555] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:26:45 2022] PUT /identity/v3/projects/7b7def58339d4553a08c29ab7683a630/users/68db9045a1414a208b0e69ddab7671e4/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:45.727126 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469addf7-e720-4bd0-b94b-5022b4bec1fc None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:45.727483 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469addf7-e720-4bd0-b94b-5022b4bec1fc None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:45.727765 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469addf7-e720-4bd0-b94b-5022b4bec1fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:45.754000 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-469addf7-e720-4bd0-b94b-5022b4bec1fc None None] MFA Rules not processed for user `68db9045a1414a208b0e69ddab7671e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:45.755695 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-469addf7-e720-4bd0-b94b-5022b4bec1fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:45.836774 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1278/2556] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:45 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:26:45.841261 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672dbed0-17ed-47a4-94ea-8a5f50d6af56 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:45.910017 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672dbed0-17ed-47a4-94ea-8a5f50d6af56 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:45.915068 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7932cb7-99c0-4ba6-9462-b7b799fa43b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:45.915732 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7932cb7-99c0-4ba6-9462-b7b799fa43b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:45.916020 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7932cb7-99c0-4ba6-9462-b7b799fa43b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:45.916300 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7932cb7-99c0-4ba6-9462-b7b799fa43b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:45.917004 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7932cb7-99c0-4ba6-9462-b7b799fa43b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:45.918899 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7932cb7-99c0-4ba6-9462-b7b799fa43b0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:45.950281 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1279/2557] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:26:45 2022] POST /identity/v3/projects => generated 389 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:26:45.954228 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469addf7-e720-4bd0-b94b-5022b4bec1fc None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:45.968355 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469addf7-e720-4bd0-b94b-5022b4bec1fc None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:46.031017 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978850c3-4bb6-4807-8421-0e0578face48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:46.031706 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978850c3-4bb6-4807-8421-0e0578face48 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:46.031995 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978850c3-4bb6-4807-8421-0e0578face48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:46.032274 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978850c3-4bb6-4807-8421-0e0578face48 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:46.032978 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978850c3-4bb6-4807-8421-0e0578face48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:46.035060 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978850c3-4bb6-4807-8421-0e0578face48 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:46.065682 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1279/2558] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:26:45 2022] POST /identity/v3/users => generated 301 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:26:46.070140 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7932cb7-99c0-4ba6-9462-b7b799fa43b0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:46.085730 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7932cb7-99c0-4ba6-9462-b7b799fa43b0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:46.092905 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793b273e-cd19-4ed8-8403-648959c8384e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:46.093624 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793b273e-cd19-4ed8-8403-648959c8384e admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:46.094170 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793b273e-cd19-4ed8-8403-648959c8384e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:46.094591 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793b273e-cd19-4ed8-8403-648959c8384e admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:46.095179 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793b273e-cd19-4ed8-8403-648959c8384e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:46.099389 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793b273e-cd19-4ed8-8403-648959c8384e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:46.110942 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1280/2559] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:26:46 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:26:46.116552 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978850c3-4bb6-4807-8421-0e0578face48 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:46.127662 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978850c3-4bb6-4807-8421-0e0578face48 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:46.133677 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367b691d-e16e-40c6-a589-2bab07fafa3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:46.134588 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367b691d-e16e-40c6-a589-2bab07fafa3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:46.134934 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367b691d-e16e-40c6-a589-2bab07fafa3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:46.135240 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367b691d-e16e-40c6-a589-2bab07fafa3e admin admin] PATH_INFO: `/v3/projects/f36e9133c8a4493bb5b3f3b9352b5c65/users/e00838a47e764669885e1ca77ca94042/roles/f36e053b442540f08041ac7878760f65` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:46.151670 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367b691d-e16e-40c6-a589-2bab07fafa3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f36e9133c8a4493bb5b3f3b9352b5c65, user_id=e00838a47e764669885e1ca77ca94042, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:46.154429 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367b691d-e16e-40c6-a589-2bab07fafa3e admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:46.170963 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1280/2560] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:26:46 2022] PUT /identity/v3/projects/f36e9133c8a4493bb5b3f3b9352b5c65/users/e00838a47e764669885e1ca77ca94042/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:46.175033 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793b273e-cd19-4ed8-8403-648959c8384e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:46.185599 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793b273e-cd19-4ed8-8403-648959c8384e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:46.235645 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:46.249270 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5568c0f3-a47d-47bc-90ec-7d05c5937074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:46.249941 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5568c0f3-a47d-47bc-90ec-7d05c5937074 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:46.250229 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5568c0f3-a47d-47bc-90ec-7d05c5937074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:46.250507 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5568c0f3-a47d-47bc-90ec-7d05c5937074 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:46.251223 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5568c0f3-a47d-47bc-90ec-7d05c5937074 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:46.252799 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5568c0f3-a47d-47bc-90ec-7d05c5937074 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:46.262335 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1281/2561] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:26:46 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:26:46.266635 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367b691d-e16e-40c6-a589-2bab07fafa3e admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:46.278140 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367b691d-e16e-40c6-a589-2bab07fafa3e admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:46.282781 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7744a0-8389-4cef-b896-c9eb5224de4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:46.283669 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7744a0-8389-4cef-b896-c9eb5224de4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:46.283957 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7744a0-8389-4cef-b896-c9eb5224de4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:46.284235 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7744a0-8389-4cef-b896-c9eb5224de4b admin admin] PATH_INFO: `/v3/projects/f36e9133c8a4493bb5b3f3b9352b5c65/users/e00838a47e764669885e1ca77ca94042/roles/f36e053b442540f08041ac7878760f65` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:46.286899 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7744a0-8389-4cef-b896-c9eb5224de4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f36e9133c8a4493bb5b3f3b9352b5c65, user_id=e00838a47e764669885e1ca77ca94042, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:46.289744 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7744a0-8389-4cef-b896-c9eb5224de4b admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:46.299889 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1281/2562] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:26:46 2022] PUT /identity/v3/projects/f36e9133c8a4493bb5b3f3b9352b5c65/users/e00838a47e764669885e1ca77ca94042/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:26:46.310162 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f236ad-2980-4a3a-b1d3-aaa249046121 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:46.310456 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f236ad-2980-4a3a-b1d3-aaa249046121 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:46.310762 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f236ad-2980-4a3a-b1d3-aaa249046121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:46.338193 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b9f236ad-2980-4a3a-b1d3-aaa249046121 None None] MFA Rules not processed for user `e00838a47e764669885e1ca77ca94042`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:46.339444 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b9f236ad-2980-4a3a-b1d3-aaa249046121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:46.414249 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1282/2563] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:46 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:26:46.424844 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7744a0-8389-4cef-b896-c9eb5224de4b admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:46.426614 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4f7744a0-8389-4cef-b896-c9eb5224de4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:46.460603 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:46.488630 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7744a0-8389-4cef-b896-c9eb5224de4b admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:46.492440 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8f42da-ebf2-40bb-90b0-4c0217855bc0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:46.493268 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f42da-ebf2-40bb-90b0-4c0217855bc0 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:46.493583 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f42da-ebf2-40bb-90b0-4c0217855bc0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:46.493931 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f42da-ebf2-40bb-90b0-4c0217855bc0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:46.503716 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8f42da-ebf2-40bb-90b0-4c0217855bc0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:46.505346 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8f42da-ebf2-40bb-90b0-4c0217855bc0 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:46.514928 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1282/2564] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:26:46 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:26:46.652688 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f236ad-2980-4a3a-b1d3-aaa249046121 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:46.729816 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f236ad-2980-4a3a-b1d3-aaa249046121 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:46.734363 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625b15f9-e9a5-42e6-84a9-f6fdcf6af581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:46.735248 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625b15f9-e9a5-42e6-84a9-f6fdcf6af581 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:46.735532 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625b15f9-e9a5-42e6-84a9-f6fdcf6af581 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:46.735810 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625b15f9-e9a5-42e6-84a9-f6fdcf6af581 admin admin] PATH_INFO: `/v3/users/68db9045a1414a208b0e69ddab7671e4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:46.736994 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625b15f9-e9a5-42e6-84a9-f6fdcf6af581 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68db9045a1414a208b0e69ddab7671e4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:46.738975 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625b15f9-e9a5-42e6-84a9-f6fdcf6af581 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:46.790102 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-625b15f9-e9a5-42e6-84a9-f6fdcf6af581 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68db9045a1414a208b0e69ddab7671e4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:46.790757 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-625b15f9-e9a5-42e6-84a9-f6fdcf6af581 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68db9045a1414a208b0e69ddab7671e4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:46.798686 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-625b15f9-e9a5-42e6-84a9-f6fdcf6af581 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68db9045a1414a208b0e69ddab7671e4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:46.812910 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-625b15f9-e9a5-42e6-84a9-f6fdcf6af581 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68db9045a1414a208b0e69ddab7671e4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:46.813796 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-625b15f9-e9a5-42e6-84a9-f6fdcf6af581 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68db9045a1414a208b0e69ddab7671e4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:46.814959 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:46.835143 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1283/2565] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:26:46 2022] DELETE /identity/v3/users/68db9045a1414a208b0e69ddab7671e4 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:46.838892 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a17b1-f244-4641-94e5-17eb698aa0af None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:46.839387 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a17b1-f244-4641-94e5-17eb698aa0af None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:46.839589 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a17b1-f244-4641-94e5-17eb698aa0af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:46.859901 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-7f6a17b1-f244-4641-94e5-17eb698aa0af None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:46.861159 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7f6a17b1-f244-4641-94e5-17eb698aa0af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:46.931468 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1283/2566] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:26:46.940990 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625b15f9-e9a5-42e6-84a9-f6fdcf6af581 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:46.942566 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-625b15f9-e9a5-42e6-84a9-f6fdcf6af581 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:47.010017 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625b15f9-e9a5-42e6-84a9-f6fdcf6af581 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:47.013830 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13226be-bc62-4343-b87e-1a8ee7a62b7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:47.014492 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13226be-bc62-4343-b87e-1a8ee7a62b7d service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:47.014778 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13226be-bc62-4343-b87e-1a8ee7a62b7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:47.015057 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13226be-bc62-4343-b87e-1a8ee7a62b7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:47.025832 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13226be-bc62-4343-b87e-1a8ee7a62b7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:47.027269 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13226be-bc62-4343-b87e-1a8ee7a62b7d service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:47.039678 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1284/2567] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:26:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:26:47.361876 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6a17b1-f244-4641-94e5-17eb698aa0af None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:47.363555 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7f6a17b1-f244-4641-94e5-17eb698aa0af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:47.430037 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6a17b1-f244-4641-94e5-17eb698aa0af None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:47.435537 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f24c5d-c08e-41ca-a38b-87f380d0242b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:47.436431 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f24c5d-c08e-41ca-a38b-87f380d0242b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:47.436747 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f24c5d-c08e-41ca-a38b-87f380d0242b admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:47.436992 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f24c5d-c08e-41ca-a38b-87f380d0242b admin admin] PATH_INFO: `/v3/projects/7b7def58339d4553a08c29ab7683a630` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:47.438417 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f24c5d-c08e-41ca-a38b-87f380d0242b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b7def58339d4553a08c29ab7683a630)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:47.440351 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f24c5d-c08e-41ca-a38b-87f380d0242b admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:47.527580 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-19f24c5d-c08e-41ca-a38b-87f380d0242b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b7def58339d4553a08c29ab7683a630', 'request_id': 'req-19f24c5d-c08e-41ca-a38b-87f380d0242b'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:47.543064 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-19f24c5d-c08e-41ca-a38b-87f380d0242b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b7def58339d4553a08c29ab7683a630', 'request_id': 'req-19f24c5d-c08e-41ca-a38b-87f380d0242b'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:47.553595 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-19f24c5d-c08e-41ca-a38b-87f380d0242b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b7def58339d4553a08c29ab7683a630', 'request_id': 'req-19f24c5d-c08e-41ca-a38b-87f380d0242b'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:47.574893 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-19f24c5d-c08e-41ca-a38b-87f380d0242b admin admin] The token cache is being invalidate because project 7b7def58339d4553a08c29ab7683a630 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:26:47.575408 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-19f24c5d-c08e-41ca-a38b-87f380d0242b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:47.576352 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-19f24c5d-c08e-41ca-a38b-87f380d0242b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:47.578535 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1284/2568] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:26:47 2022] DELETE /identity/v3/projects/7b7def58339d4553a08c29ab7683a630 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:47.582224 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13226be-bc62-4343-b87e-1a8ee7a62b7d service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:47.603368 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13226be-bc62-4343-b87e-1a8ee7a62b7d service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:47.666793 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5326912a-225e-43b3-8d54-525d28d4435e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:47.667745 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5326912a-225e-43b3-8d54-525d28d4435e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:47.668130 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5326912a-225e-43b3-8d54-525d28d4435e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:47.668470 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5326912a-225e-43b3-8d54-525d28d4435e admin admin] PATH_INFO: `/v3/users/e00838a47e764669885e1ca77ca94042` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:47.669798 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5326912a-225e-43b3-8d54-525d28d4435e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e00838a47e764669885e1ca77ca94042)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:47.674455 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5326912a-225e-43b3-8d54-525d28d4435e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:47.730442 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5326912a-225e-43b3-8d54-525d28d4435e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e00838a47e764669885e1ca77ca94042'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:47.731112 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5326912a-225e-43b3-8d54-525d28d4435e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e00838a47e764669885e1ca77ca94042'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:47.738529 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5326912a-225e-43b3-8d54-525d28d4435e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e00838a47e764669885e1ca77ca94042'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:47.754228 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5326912a-225e-43b3-8d54-525d28d4435e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e00838a47e764669885e1ca77ca94042'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:47.755655 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5326912a-225e-43b3-8d54-525d28d4435e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e00838a47e764669885e1ca77ca94042'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:47.768370 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1285/2569] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:26:47 2022] DELETE /identity/v3/users/e00838a47e764669885e1ca77ca94042 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:48.065662 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f24c5d-c08e-41ca-a38b-87f380d0242b admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:48.067197 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-19f24c5d-c08e-41ca-a38b-87f380d0242b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:48.081674 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f24c5d-c08e-41ca-a38b-87f380d0242b admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:48.142796 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:48.150974 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f75cfb-e792-4a45-a93c-764f066a6f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:48.151967 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f75cfb-e792-4a45-a93c-764f066a6f20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:48.152369 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f75cfb-e792-4a45-a93c-764f066a6f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:48.152671 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f75cfb-e792-4a45-a93c-764f066a6f20 admin admin] PATH_INFO: `/v3/projects/f36e9133c8a4493bb5b3f3b9352b5c65` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:48.154743 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f75cfb-e792-4a45-a93c-764f066a6f20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f36e9133c8a4493bb5b3f3b9352b5c65)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:48.157990 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f75cfb-e792-4a45-a93c-764f066a6f20 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:48.215502 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-62f75cfb-e792-4a45-a93c-764f066a6f20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f36e9133c8a4493bb5b3f3b9352b5c65', 'request_id': 'req-62f75cfb-e792-4a45-a93c-764f066a6f20'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:48.221393 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-62f75cfb-e792-4a45-a93c-764f066a6f20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f36e9133c8a4493bb5b3f3b9352b5c65', 'request_id': 'req-62f75cfb-e792-4a45-a93c-764f066a6f20'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:48.228169 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-62f75cfb-e792-4a45-a93c-764f066a6f20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f36e9133c8a4493bb5b3f3b9352b5c65', 'request_id': 'req-62f75cfb-e792-4a45-a93c-764f066a6f20'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:48.242567 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-62f75cfb-e792-4a45-a93c-764f066a6f20 admin admin] The token cache is being invalidate because project f36e9133c8a4493bb5b3f3b9352b5c65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:26:48.242866 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-62f75cfb-e792-4a45-a93c-764f066a6f20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:48.243585 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-62f75cfb-e792-4a45-a93c-764f066a6f20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:48.245334 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1285/2570] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:26:48 2022] DELETE /identity/v3/projects/f36e9133c8a4493bb5b3f3b9352b5c65 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:48.250234 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a7920-d679-4c63-9f14-fd3318aab062 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:48.250608 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a7920-d679-4c63-9f14-fd3318aab062 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:48.251260 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a7920-d679-4c63-9f14-fd3318aab062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:48.274635 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-170a7920-d679-4c63-9f14-fd3318aab062 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:48.276647 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-170a7920-d679-4c63-9f14-fd3318aab062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:48.282317 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:48.342271 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1286/2571] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:26:48.351107 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e3acc3-0c4f-4d9a-9104-2a292e251cc2 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:48.351573 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e3acc3-0c4f-4d9a-9104-2a292e251cc2 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:48.351970 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e3acc3-0c4f-4d9a-9104-2a292e251cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:48.373146 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-f4e3acc3-0c4f-4d9a-9104-2a292e251cc2 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:48.374542 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f4e3acc3-0c4f-4d9a-9104-2a292e251cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:48.382917 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1286/2572] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:26:48.387253 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170a7920-d679-4c63-9f14-fd3318aab062 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:48.397386 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170a7920-d679-4c63-9f14-fd3318aab062 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:48.403009 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8966874f-b5fb-4131-96a9-caad00322907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:48.403009 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8966874f-b5fb-4131-96a9-caad00322907 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:48.403275 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8966874f-b5fb-4131-96a9-caad00322907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:48.403408 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8966874f-b5fb-4131-96a9-caad00322907 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:48.404191 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8966874f-b5fb-4131-96a9-caad00322907 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:26:48.404519 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8966874f-b5fb-4131-96a9-caad00322907 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:48.409575 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8966874f-b5fb-4131-96a9-caad00322907 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:48.419352 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1287/2573] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:26:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:26:48.423923 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e3acc3-0c4f-4d9a-9104-2a292e251cc2 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:48.434709 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e3acc3-0c4f-4d9a-9104-2a292e251cc2 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:48.439269 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e5b755-7d73-4d7f-88e7-4c4ce2d5abfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:48.439931 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e5b755-7d73-4d7f-88e7-4c4ce2d5abfa admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:48.440216 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e5b755-7d73-4d7f-88e7-4c4ce2d5abfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:48.440504 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e5b755-7d73-4d7f-88e7-4c4ce2d5abfa admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:48.441247 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e5b755-7d73-4d7f-88e7-4c4ce2d5abfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:48.443187 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e5b755-7d73-4d7f-88e7-4c4ce2d5abfa admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:48.477767 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1287/2574] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:26:48 2022] POST /identity/v3/projects => generated 379 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:26:48.479039 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8966874f-b5fb-4131-96a9-caad00322907 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:48.489830 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8966874f-b5fb-4131-96a9-caad00322907 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:48.555313 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49bb6e2-e06e-477e-ae7b-eab6b7f165e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:48.556005 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49bb6e2-e06e-477e-ae7b-eab6b7f165e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:48.556290 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49bb6e2-e06e-477e-ae7b-eab6b7f165e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:48.556568 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49bb6e2-e06e-477e-ae7b-eab6b7f165e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:48.557298 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49bb6e2-e06e-477e-ae7b-eab6b7f165e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:48.559205 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49bb6e2-e06e-477e-ae7b-eab6b7f165e4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:48.580317 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1288/2575] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:26:48 2022] POST /identity/v3/users => generated 290 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:26:48.584764 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e5b755-7d73-4d7f-88e7-4c4ce2d5abfa admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:48.595632 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e5b755-7d73-4d7f-88e7-4c4ce2d5abfa admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:48.604622 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78001940-5878-446e-a330-fe88c39cdc88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:48.605457 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78001940-5878-446e-a330-fe88c39cdc88 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:48.605745 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78001940-5878-446e-a330-fe88c39cdc88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:48.606022 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78001940-5878-446e-a330-fe88c39cdc88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:48.606763 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78001940-5878-446e-a330-fe88c39cdc88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:48.608519 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78001940-5878-446e-a330-fe88c39cdc88 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:48.626455 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49bb6e2-e06e-477e-ae7b-eab6b7f165e4 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:48.627092 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1288/2576] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:26:48 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:26:48.639030 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49bb6e2-e06e-477e-ae7b-eab6b7f165e4 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:48.652137 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991c6b65-1bde-4cf6-832e-35dbdb964ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:48.653095 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c6b65-1bde-4cf6-832e-35dbdb964ac8 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:48.653450 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c6b65-1bde-4cf6-832e-35dbdb964ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:48.653906 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c6b65-1bde-4cf6-832e-35dbdb964ac8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:48.654858 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991c6b65-1bde-4cf6-832e-35dbdb964ac8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:48.656596 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991c6b65-1bde-4cf6-832e-35dbdb964ac8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:48.669882 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1289/2577] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:26:48 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:26:48.682618 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78001940-5878-446e-a330-fe88c39cdc88 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:48.684850 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78001940-5878-446e-a330-fe88c39cdc88 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:48.695385 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15c6207-0097-4704-87e1-965ee4d4974c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:48.696195 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c6207-0097-4704-87e1-965ee4d4974c admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:48.696552 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c6207-0097-4704-87e1-965ee4d4974c admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:48.696891 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c6207-0097-4704-87e1-965ee4d4974c admin admin] PATH_INFO: `/v3/projects/d6da64bfc98242f4af4ecf69f60617f2/users/136a6f84df3a47a0876d1bd3fb0c2317/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:48.718078 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15c6207-0097-4704-87e1-965ee4d4974c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6da64bfc98242f4af4ecf69f60617f2, user_id=136a6f84df3a47a0876d1bd3fb0c2317, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:48.721426 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15c6207-0097-4704-87e1-965ee4d4974c admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:48.749285 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1289/2578] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:26:48 2022] PUT /identity/v3/projects/d6da64bfc98242f4af4ecf69f60617f2/users/136a6f84df3a47a0876d1bd3fb0c2317/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:48.757711 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a15a4-4892-47e3-aaeb-1141982c381a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:48.758005 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a15a4-4892-47e3-aaeb-1141982c381a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:48.758280 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a15a4-4892-47e3-aaeb-1141982c381a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:48.795793 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-670a15a4-4892-47e3-aaeb-1141982c381a None None] MFA Rules not processed for user `136a6f84df3a47a0876d1bd3fb0c2317`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:48.797079 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-670a15a4-4892-47e3-aaeb-1141982c381a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:48.888626 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1290/2579] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:48 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:26:48.890877 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15c6207-0097-4704-87e1-965ee4d4974c admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:48.967500 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15c6207-0097-4704-87e1-965ee4d4974c admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:48.972534 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aaccc5-8928-4809-ae51-21b4d5edafaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:48.973272 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aaccc5-8928-4809-ae51-21b4d5edafaa admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:48.973707 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aaccc5-8928-4809-ae51-21b4d5edafaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:48.974041 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aaccc5-8928-4809-ae51-21b4d5edafaa admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:48.974765 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aaccc5-8928-4809-ae51-21b4d5edafaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:48.976813 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aaccc5-8928-4809-ae51-21b4d5edafaa admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:49.009293 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1290/2580] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:26:48 2022] POST /identity/v3/projects => generated 377 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:26:49.010472 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670a15a4-4892-47e3-aaeb-1141982c381a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:49.022499 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670a15a4-4892-47e3-aaeb-1141982c381a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:49.026194 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:49.088527 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dd911a-214f-4ed6-b428-71f60a79b7f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:49.089339 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dd911a-214f-4ed6-b428-71f60a79b7f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:49.089628 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dd911a-214f-4ed6-b428-71f60a79b7f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:49.090272 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dd911a-214f-4ed6-b428-71f60a79b7f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:49.090984 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dd911a-214f-4ed6-b428-71f60a79b7f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:49.092940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dd911a-214f-4ed6-b428-71f60a79b7f3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:49.123110 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1291/2581] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:26:49 2022] POST /identity/v3/users => generated 289 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:26:49.124595 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aaccc5-8928-4809-ae51-21b4d5edafaa admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:49.135280 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aaccc5-8928-4809-ae51-21b4d5edafaa admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:49.140002 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045407b6-fc8d-4402-9660-dbf583837931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:49.140673 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045407b6-fc8d-4402-9660-dbf583837931 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:49.140963 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045407b6-fc8d-4402-9660-dbf583837931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:49.141270 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045407b6-fc8d-4402-9660-dbf583837931 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:49.142007 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045407b6-fc8d-4402-9660-dbf583837931 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:49.143605 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045407b6-fc8d-4402-9660-dbf583837931 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:49.144981 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:49.157987 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1291/2582] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:26:49.162457 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dd911a-214f-4ed6-b428-71f60a79b7f3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:49.174527 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dd911a-214f-4ed6-b428-71f60a79b7f3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:49.179872 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea518f6c-c228-468e-a43d-6ee10f140527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:49.180541 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea518f6c-c228-468e-a43d-6ee10f140527 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:49.180825 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea518f6c-c228-468e-a43d-6ee10f140527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:49.181132 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea518f6c-c228-468e-a43d-6ee10f140527 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:49.181854 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea518f6c-c228-468e-a43d-6ee10f140527 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:49.183493 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea518f6c-c228-468e-a43d-6ee10f140527 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:49.193987 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1292/2583] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:26:49.198141 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045407b6-fc8d-4402-9660-dbf583837931 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:49.209413 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045407b6-fc8d-4402-9660-dbf583837931 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:49.214365 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed521e45-aac0-42be-8895-b76b9bab78b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:49.215262 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed521e45-aac0-42be-8895-b76b9bab78b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:49.215546 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed521e45-aac0-42be-8895-b76b9bab78b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:49.215829 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed521e45-aac0-42be-8895-b76b9bab78b4 admin admin] PATH_INFO: `/v3/projects/381308b3c3384fa2a896d848445caff6/users/833d5a1376394cad8015c8340aeec9e8/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:49.231863 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed521e45-aac0-42be-8895-b76b9bab78b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=381308b3c3384fa2a896d848445caff6, user_id=833d5a1376394cad8015c8340aeec9e8, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:49.234627 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed521e45-aac0-42be-8895-b76b9bab78b4 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:49.247525 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1292/2584] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:26:49 2022] PUT /identity/v3/projects/381308b3c3384fa2a896d848445caff6/users/833d5a1376394cad8015c8340aeec9e8/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:49.257770 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f6fc56-6712-4396-b98f-4cf77abfcf6e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:49.258061 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f6fc56-6712-4396-b98f-4cf77abfcf6e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:49.258372 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f6fc56-6712-4396-b98f-4cf77abfcf6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:49.284773 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-99f6fc56-6712-4396-b98f-4cf77abfcf6e None None] MFA Rules not processed for user `833d5a1376394cad8015c8340aeec9e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:49.286113 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-99f6fc56-6712-4396-b98f-4cf77abfcf6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:49.349776 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1293/2585] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:49 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:26:49.354338 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed521e45-aac0-42be-8895-b76b9bab78b4 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:49.416394 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed521e45-aac0-42be-8895-b76b9bab78b4 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:49.420969 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31976ef3-4520-47b2-bafe-6666f79431ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:49.421694 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31976ef3-4520-47b2-bafe-6666f79431ab admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:49.421982 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31976ef3-4520-47b2-bafe-6666f79431ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:49.422260 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31976ef3-4520-47b2-bafe-6666f79431ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:49.422943 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31976ef3-4520-47b2-bafe-6666f79431ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:49.424840 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31976ef3-4520-47b2-bafe-6666f79431ab admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:49.449760 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1293/2586] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:26:49 2022] POST /identity/v3/projects => generated 379 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:26:49.453857 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f6fc56-6712-4396-b98f-4cf77abfcf6e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:49.468371 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f6fc56-6712-4396-b98f-4cf77abfcf6e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:49.527101 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96c88e4-b5ca-490c-965a-8725a6042f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:49.527700 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c88e4-b5ca-490c-965a-8725a6042f3a admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:49.527985 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c88e4-b5ca-490c-965a-8725a6042f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:49.528266 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c88e4-b5ca-490c-965a-8725a6042f3a admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:49.528966 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96c88e4-b5ca-490c-965a-8725a6042f3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:49.530889 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96c88e4-b5ca-490c-965a-8725a6042f3a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:49.551064 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1294/2587] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:26:49 2022] POST /identity/v3/users => generated 296 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:26:49.555077 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31976ef3-4520-47b2-bafe-6666f79431ab admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:49.565456 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31976ef3-4520-47b2-bafe-6666f79431ab admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:49.570796 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2b12be-3fa7-40b0-b8ed-60b614dd5e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:49.571451 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b12be-3fa7-40b0-b8ed-60b614dd5e13 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:49.571736 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b12be-3fa7-40b0-b8ed-60b614dd5e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:49.572013 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b12be-3fa7-40b0-b8ed-60b614dd5e13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:49.572733 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2b12be-3fa7-40b0-b8ed-60b614dd5e13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:49.574367 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2b12be-3fa7-40b0-b8ed-60b614dd5e13 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:49.585542 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1294/2588] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:26:49.589537 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96c88e4-b5ca-490c-965a-8725a6042f3a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:49.595910 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:49.600066 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96c88e4-b5ca-490c-965a-8725a6042f3a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:49.605806 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8672e99a-9ab0-4d4d-b8b0-5e9f96df919b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:49.606691 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8672e99a-9ab0-4d4d-b8b0-5e9f96df919b admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:49.607001 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8672e99a-9ab0-4d4d-b8b0-5e9f96df919b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:49.607284 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8672e99a-9ab0-4d4d-b8b0-5e9f96df919b admin admin] PATH_INFO: `/v3/projects/a4e985abf5714d8cb0552c97c1a22db7/users/dc855cff70a1467d828b9d051e61588a/roles/f36e053b442540f08041ac7878760f65` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:49.623224 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8672e99a-9ab0-4d4d-b8b0-5e9f96df919b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4e985abf5714d8cb0552c97c1a22db7, user_id=dc855cff70a1467d828b9d051e61588a, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:49.625909 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8672e99a-9ab0-4d4d-b8b0-5e9f96df919b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:49.638322 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1295/2589] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:26:49 2022] PUT /identity/v3/projects/a4e985abf5714d8cb0552c97c1a22db7/users/dc855cff70a1467d828b9d051e61588a/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:26:49.642358 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2b12be-3fa7-40b0-b8ed-60b614dd5e13 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:49.653142 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2b12be-3fa7-40b0-b8ed-60b614dd5e13 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:49.707705 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd21502-3ffc-4235-ae69-5ced2edc9439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:49.709087 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd21502-3ffc-4235-ae69-5ced2edc9439 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:49.709438 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd21502-3ffc-4235-ae69-5ced2edc9439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:49.709743 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd21502-3ffc-4235-ae69-5ced2edc9439 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:49.710572 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd21502-3ffc-4235-ae69-5ced2edc9439 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:49.712179 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd21502-3ffc-4235-ae69-5ced2edc9439 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:49.722323 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1295/2590] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:26:49.726481 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8672e99a-9ab0-4d4d-b8b0-5e9f96df919b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:49.736744 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8672e99a-9ab0-4d4d-b8b0-5e9f96df919b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:49.741341 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f2553d-33f0-4881-ba7f-2ee0a4fffe20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:49.742208 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f2553d-33f0-4881-ba7f-2ee0a4fffe20 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:49.742490 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f2553d-33f0-4881-ba7f-2ee0a4fffe20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:49.742761 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f2553d-33f0-4881-ba7f-2ee0a4fffe20 admin admin] PATH_INFO: `/v3/projects/a4e985abf5714d8cb0552c97c1a22db7/users/dc855cff70a1467d828b9d051e61588a/roles/f36e053b442540f08041ac7878760f65` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:49.744724 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f2553d-33f0-4881-ba7f-2ee0a4fffe20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4e985abf5714d8cb0552c97c1a22db7, user_id=dc855cff70a1467d828b9d051e61588a, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:49.747328 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f2553d-33f0-4881-ba7f-2ee0a4fffe20 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:49.758906 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1296/2591] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:26:49 2022] PUT /identity/v3/projects/a4e985abf5714d8cb0552c97c1a22db7/users/dc855cff70a1467d828b9d051e61588a/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:49.769671 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724fb44e-a45f-4061-b1da-6d87df05a01d None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:49.769968 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724fb44e-a45f-4061-b1da-6d87df05a01d None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:49.770254 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724fb44e-a45f-4061-b1da-6d87df05a01d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:49.797000 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-724fb44e-a45f-4061-b1da-6d87df05a01d None None] MFA Rules not processed for user `dc855cff70a1467d828b9d051e61588a`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:49.798187 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-724fb44e-a45f-4061-b1da-6d87df05a01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:49.829062 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:49.871470 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1296/2592] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:49 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:26:49.883304 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f2553d-33f0-4881-ba7f-2ee0a4fffe20 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:49.885020 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-24f2553d-33f0-4881-ba7f-2ee0a4fffe20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:49.941906 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f2553d-33f0-4881-ba7f-2ee0a4fffe20 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:49.946199 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e82e812-bf4c-4a5b-a5cd-da925a83012b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:49.946848 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82e812-bf4c-4a5b-a5cd-da925a83012b service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:49.947135 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82e812-bf4c-4a5b-a5cd-da925a83012b service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:49.947432 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82e812-bf4c-4a5b-a5cd-da925a83012b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:49.957066 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e82e812-bf4c-4a5b-a5cd-da925a83012b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:49.958536 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e82e812-bf4c-4a5b-a5cd-da925a83012b service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:49.968313 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1297/2593] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:26:49 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:26:50.007363 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724fb44e-a45f-4061-b1da-6d87df05a01d None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:50.008778 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-724fb44e-a45f-4061-b1da-6d87df05a01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:50.021064 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724fb44e-a45f-4061-b1da-6d87df05a01d None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:50.024687 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f57c650-95ef-4f01-a55e-ad68fc552bc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:50.025365 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57c650-95ef-4f01-a55e-ad68fc552bc1 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:50.025651 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57c650-95ef-4f01-a55e-ad68fc552bc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:50.025932 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57c650-95ef-4f01-a55e-ad68fc552bc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:50.082343 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f57c650-95ef-4f01-a55e-ad68fc552bc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:50.083905 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f57c650-95ef-4f01-a55e-ad68fc552bc1 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:50.092854 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1297/2594] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:26:50 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:26:54.008723 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e82e812-bf4c-4a5b-a5cd-da925a83012b service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:54.073325 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e82e812-bf4c-4a5b-a5cd-da925a83012b service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:54.078211 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0ee91-2281-4636-8061-e5dea0e7b51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:54.078958 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0ee91-2281-4636-8061-e5dea0e7b51c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:54.079272 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0ee91-2281-4636-8061-e5dea0e7b51c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:54.079583 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0ee91-2281-4636-8061-e5dea0e7b51c admin admin] PATH_INFO: `/v3/users/136a6f84df3a47a0876d1bd3fb0c2317` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:54.080790 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e0ee91-2281-4636-8061-e5dea0e7b51c admin admin] RBAC: Authorizing `identity:delete_user(user_id=136a6f84df3a47a0876d1bd3fb0c2317)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:54.082725 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e0ee91-2281-4636-8061-e5dea0e7b51c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:54.132689 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-93e0ee91-2281-4636-8061-e5dea0e7b51c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '136a6f84df3a47a0876d1bd3fb0c2317'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:54.133621 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-93e0ee91-2281-4636-8061-e5dea0e7b51c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '136a6f84df3a47a0876d1bd3fb0c2317'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:54.141806 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-93e0ee91-2281-4636-8061-e5dea0e7b51c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '136a6f84df3a47a0876d1bd3fb0c2317'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:54.155232 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-93e0ee91-2281-4636-8061-e5dea0e7b51c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '136a6f84df3a47a0876d1bd3fb0c2317'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:54.155894 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-93e0ee91-2281-4636-8061-e5dea0e7b51c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '136a6f84df3a47a0876d1bd3fb0c2317'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:54.157221 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:54.171143 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1298/2595] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:26:54 2022] DELETE /identity/v3/users/136a6f84df3a47a0876d1bd3fb0c2317 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:54.175709 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba96d89-8664-4a26-aeec-de398c5b9d24 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:54.176174 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba96d89-8664-4a26-aeec-de398c5b9d24 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:54.176290 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba96d89-8664-4a26-aeec-de398c5b9d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:54.194274 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-0ba96d89-8664-4a26-aeec-de398c5b9d24 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:54.195500 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0ba96d89-8664-4a26-aeec-de398c5b9d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:54.258902 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1298/2596] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:26:54.268528 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0ee91-2281-4636-8061-e5dea0e7b51c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:54.270082 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-93e0ee91-2281-4636-8061-e5dea0e7b51c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:54.329393 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0ee91-2281-4636-8061-e5dea0e7b51c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:54.333907 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189fd647-3538-4826-822e-4c2cf6f5bd49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:54.334635 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189fd647-3538-4826-822e-4c2cf6f5bd49 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:54.334923 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189fd647-3538-4826-822e-4c2cf6f5bd49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:54.335236 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189fd647-3538-4826-822e-4c2cf6f5bd49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:54.345571 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189fd647-3538-4826-822e-4c2cf6f5bd49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:54.347018 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189fd647-3538-4826-822e-4c2cf6f5bd49 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:54.356774 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1299/2597] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:26:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:26:54.635178 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba96d89-8664-4a26-aeec-de398c5b9d24 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:54.637391 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0ba96d89-8664-4a26-aeec-de398c5b9d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:54.652290 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba96d89-8664-4a26-aeec-de398c5b9d24 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:54.658671 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c0444e-da05-4638-9e6f-77c5679a4a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:54.662609 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c0444e-da05-4638-9e6f-77c5679a4a24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:54.662879 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c0444e-da05-4638-9e6f-77c5679a4a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:54.663205 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c0444e-da05-4638-9e6f-77c5679a4a24 admin admin] PATH_INFO: `/v3/users/bfd08eca01724dd2b8ea0d4196692b80` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:54.664622 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c0444e-da05-4638-9e6f-77c5679a4a24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfd08eca01724dd2b8ea0d4196692b80)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:54.666712 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c0444e-da05-4638-9e6f-77c5679a4a24 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:54.674392 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189fd647-3538-4826-822e-4c2cf6f5bd49 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:54.676045 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-189fd647-3538-4826-822e-4c2cf6f5bd49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:54.711340 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189fd647-3538-4826-822e-4c2cf6f5bd49 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:54.720319 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7d0985-b85f-4b9e-a11f-b7c9c9e0a966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:54.721480 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7d0985-b85f-4b9e-a11f-b7c9c9e0a966 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:54.721826 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7d0985-b85f-4b9e-a11f-b7c9c9e0a966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:54.722115 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7d0985-b85f-4b9e-a11f-b7c9c9e0a966 admin admin] PATH_INFO: `/v3/projects/d6da64bfc98242f4af4ecf69f60617f2` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:54.723696 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7d0985-b85f-4b9e-a11f-b7c9c9e0a966 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6da64bfc98242f4af4ecf69f60617f2)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:54.726370 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7d0985-b85f-4b9e-a11f-b7c9c9e0a966 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:54.735222 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a7c0444e-da05-4638-9e6f-77c5679a4a24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfd08eca01724dd2b8ea0d4196692b80'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:54.744783 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a7c0444e-da05-4638-9e6f-77c5679a4a24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfd08eca01724dd2b8ea0d4196692b80'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:54.745525 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a7c0444e-da05-4638-9e6f-77c5679a4a24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfd08eca01724dd2b8ea0d4196692b80'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:54.750963 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:54.753102 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a7c0444e-da05-4638-9e6f-77c5679a4a24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfd08eca01724dd2b8ea0d4196692b80'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:54.769425 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a7c0444e-da05-4638-9e6f-77c5679a4a24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfd08eca01724dd2b8ea0d4196692b80'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:54.776803 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1299/2598] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:26:54 2022] DELETE /identity/v3/users/bfd08eca01724dd2b8ea0d4196692b80 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:54.781391 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d07d254-156e-46ba-b72a-5b747b73b43e None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:54.781680 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d07d254-156e-46ba-b72a-5b747b73b43e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:54.782012 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d07d254-156e-46ba-b72a-5b747b73b43e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:54.784175 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ef7d0985-b85f-4b9e-a11f-b7c9c9e0a966 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6da64bfc98242f4af4ecf69f60617f2', 'request_id': 'req-ef7d0985-b85f-4b9e-a11f-b7c9c9e0a966'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:54.790750 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ef7d0985-b85f-4b9e-a11f-b7c9c9e0a966 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6da64bfc98242f4af4ecf69f60617f2', 'request_id': 'req-ef7d0985-b85f-4b9e-a11f-b7c9c9e0a966'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:54.797610 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ef7d0985-b85f-4b9e-a11f-b7c9c9e0a966 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6da64bfc98242f4af4ecf69f60617f2', 'request_id': 'req-ef7d0985-b85f-4b9e-a11f-b7c9c9e0a966'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:54.799969 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-2d07d254-156e-46ba-b72a-5b747b73b43e None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:54.801906 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-2d07d254-156e-46ba-b72a-5b747b73b43e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:54.814027 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ef7d0985-b85f-4b9e-a11f-b7c9c9e0a966 admin admin] The token cache is being invalidate because project d6da64bfc98242f4af4ecf69f60617f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:26:54.814766 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ef7d0985-b85f-4b9e-a11f-b7c9c9e0a966 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:54.815491 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ef7d0985-b85f-4b9e-a11f-b7c9c9e0a966 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:54.818023 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1300/2599] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:26:54 2022] DELETE /identity/v3/projects/d6da64bfc98242f4af4ecf69f60617f2 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:26:54.822008 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7d0985-b85f-4b9e-a11f-b7c9c9e0a966 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:54.824208 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ef7d0985-b85f-4b9e-a11f-b7c9c9e0a966 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:54.870625 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1300/2600] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:26:54.881326 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d07d254-156e-46ba-b72a-5b747b73b43e None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:54.882725 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-2d07d254-156e-46ba-b72a-5b747b73b43e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:54.907318 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7d0985-b85f-4b9e-a11f-b7c9c9e0a966 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:54.912861 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04625ac9-49a1-4ffe-b336-e4e8e0444eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:54.914015 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04625ac9-49a1-4ffe-b336-e4e8e0444eac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:54.914338 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04625ac9-49a1-4ffe-b336-e4e8e0444eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:54.914677 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04625ac9-49a1-4ffe-b336-e4e8e0444eac admin admin] PATH_INFO: `/v3/users/833d5a1376394cad8015c8340aeec9e8` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:54.916046 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04625ac9-49a1-4ffe-b336-e4e8e0444eac admin admin] RBAC: Authorizing `identity:delete_user(user_id=833d5a1376394cad8015c8340aeec9e8)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:54.918246 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04625ac9-49a1-4ffe-b336-e4e8e0444eac admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:54.936806 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:54.956459 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d07d254-156e-46ba-b72a-5b747b73b43e None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:54.961717 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239fbf4b-19e5-4c36-9c42-44062d2dceaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:54.962413 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239fbf4b-19e5-4c36-9c42-44062d2dceaf service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:54.962815 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239fbf4b-19e5-4c36-9c42-44062d2dceaf service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:54.963191 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239fbf4b-19e5-4c36-9c42-44062d2dceaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:54.973310 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239fbf4b-19e5-4c36-9c42-44062d2dceaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:54.973835 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-04625ac9-49a1-4ffe-b336-e4e8e0444eac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '833d5a1376394cad8015c8340aeec9e8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:54.974673 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-04625ac9-49a1-4ffe-b336-e4e8e0444eac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '833d5a1376394cad8015c8340aeec9e8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:54.975075 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239fbf4b-19e5-4c36-9c42-44062d2dceaf service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:54.988294 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-04625ac9-49a1-4ffe-b336-e4e8e0444eac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '833d5a1376394cad8015c8340aeec9e8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:54.992003 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1301/2601] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:26:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:26:55.006674 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-04625ac9-49a1-4ffe-b336-e4e8e0444eac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '833d5a1376394cad8015c8340aeec9e8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:55.007436 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-04625ac9-49a1-4ffe-b336-e4e8e0444eac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '833d5a1376394cad8015c8340aeec9e8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:55.020045 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1301/2602] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:26:54 2022] DELETE /identity/v3/users/833d5a1376394cad8015c8340aeec9e8 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:26:55.298539 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239fbf4b-19e5-4c36-9c42-44062d2dceaf service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:55.300034 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-239fbf4b-19e5-4c36-9c42-44062d2dceaf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:55.315256 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239fbf4b-19e5-4c36-9c42-44062d2dceaf service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:55.339523 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04625ac9-49a1-4ffe-b336-e4e8e0444eac admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:55.341066 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-04625ac9-49a1-4ffe-b336-e4e8e0444eac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:55.399453 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5602f3bf-3361-4d62-8c7d-241259450c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:55.400349 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602f3bf-3361-4d62-8c7d-241259450c64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:55.400635 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602f3bf-3361-4d62-8c7d-241259450c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:55.400921 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602f3bf-3361-4d62-8c7d-241259450c64 admin admin] PATH_INFO: `/v3/projects/60a3a514b11b4c3eb02ba537fb75d577` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:55.402170 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5602f3bf-3361-4d62-8c7d-241259450c64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60a3a514b11b4c3eb02ba537fb75d577)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:55.404127 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5602f3bf-3361-4d62-8c7d-241259450c64 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:55.430123 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04625ac9-49a1-4ffe-b336-e4e8e0444eac admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:55.430556 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:55.435603 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90f2358-343d-4054-a796-889be2ee790f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:55.436601 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f2358-343d-4054-a796-889be2ee790f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:55.436892 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f2358-343d-4054-a796-889be2ee790f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:55.437316 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f2358-343d-4054-a796-889be2ee790f admin admin] PATH_INFO: `/v3/projects/381308b3c3384fa2a896d848445caff6` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:55.438428 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90f2358-343d-4054-a796-889be2ee790f admin admin] RBAC: Authorizing `identity:delete_project(project_id=381308b3c3384fa2a896d848445caff6)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:55.440395 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90f2358-343d-4054-a796-889be2ee790f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:55.455479 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-5602f3bf-3361-4d62-8c7d-241259450c64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60a3a514b11b4c3eb02ba537fb75d577', 'request_id': 'req-5602f3bf-3361-4d62-8c7d-241259450c64'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:55.461099 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-5602f3bf-3361-4d62-8c7d-241259450c64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60a3a514b11b4c3eb02ba537fb75d577', 'request_id': 'req-5602f3bf-3361-4d62-8c7d-241259450c64'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:55.467389 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-5602f3bf-3361-4d62-8c7d-241259450c64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60a3a514b11b4c3eb02ba537fb75d577', 'request_id': 'req-5602f3bf-3361-4d62-8c7d-241259450c64'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:55.480696 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-5602f3bf-3361-4d62-8c7d-241259450c64 admin admin] The token cache is being invalidate because project 60a3a514b11b4c3eb02ba537fb75d577 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:26:55.480992 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-5602f3bf-3361-4d62-8c7d-241259450c64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:55.481714 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-5602f3bf-3361-4d62-8c7d-241259450c64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:55.483336 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1302/2603] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:26:55 2022] DELETE /identity/v3/projects/60a3a514b11b4c3eb02ba537fb75d577 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:55.488141 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1de85f-5248-4fd3-a08e-43725e589771 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:55.488513 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1de85f-5248-4fd3-a08e-43725e589771 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:55.488796 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1de85f-5248-4fd3-a08e-43725e589771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:55.490092 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f90f2358-343d-4054-a796-889be2ee790f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '381308b3c3384fa2a896d848445caff6', 'request_id': 'req-f90f2358-343d-4054-a796-889be2ee790f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:55.496231 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f90f2358-343d-4054-a796-889be2ee790f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '381308b3c3384fa2a896d848445caff6', 'request_id': 'req-f90f2358-343d-4054-a796-889be2ee790f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:55.502581 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f90f2358-343d-4054-a796-889be2ee790f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '381308b3c3384fa2a896d848445caff6', 'request_id': 'req-f90f2358-343d-4054-a796-889be2ee790f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:55.509971 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-5c1de85f-5248-4fd3-a08e-43725e589771 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:55.511258 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-5c1de85f-5248-4fd3-a08e-43725e589771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:55.518501 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f90f2358-343d-4054-a796-889be2ee790f admin admin] The token cache is being invalidate because project 381308b3c3384fa2a896d848445caff6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:26:55.518793 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f90f2358-343d-4054-a796-889be2ee790f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:55.519490 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f90f2358-343d-4054-a796-889be2ee790f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:55.521180 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1302/2604] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:26:55 2022] DELETE /identity/v3/projects/381308b3c3384fa2a896d848445caff6 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:26:55.525242 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90f2358-343d-4054-a796-889be2ee790f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:55.526698 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f90f2358-343d-4054-a796-889be2ee790f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:55.557257 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:55.581753 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1303/2605] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:26:55.589574 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ebdbad-3053-423b-9fd9-7488407351bf None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:55.589943 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ebdbad-3053-423b-9fd9-7488407351bf None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:55.590314 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ebdbad-3053-423b-9fd9-7488407351bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:55.597794 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90f2358-343d-4054-a796-889be2ee790f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:55.603187 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2605a61-97c8-41fe-b9f5-6f9c28a18982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:55.604092 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2605a61-97c8-41fe-b9f5-6f9c28a18982 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:55.604377 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2605a61-97c8-41fe-b9f5-6f9c28a18982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:55.604658 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2605a61-97c8-41fe-b9f5-6f9c28a18982 admin admin] PATH_INFO: `/v3/users/dc855cff70a1467d828b9d051e61588a` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:55.605958 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2605a61-97c8-41fe-b9f5-6f9c28a18982 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc855cff70a1467d828b9d051e61588a)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:55.607878 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2605a61-97c8-41fe-b9f5-6f9c28a18982 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:55.609699 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-15ebdbad-3053-423b-9fd9-7488407351bf None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:55.611292 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-15ebdbad-3053-423b-9fd9-7488407351bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:55.620344 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1304/2606] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:26:55.624404 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ebdbad-3053-423b-9fd9-7488407351bf None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:55.633946 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ebdbad-3053-423b-9fd9-7488407351bf None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:55.639542 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08050621-32ee-4492-812a-b9fe9bd2749e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:55.640457 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08050621-32ee-4492-812a-b9fe9bd2749e admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:55.640795 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08050621-32ee-4492-812a-b9fe9bd2749e admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:55.641146 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08050621-32ee-4492-812a-b9fe9bd2749e admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:55.642144 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08050621-32ee-4492-812a-b9fe9bd2749e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:26:55.642456 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08050621-32ee-4492-812a-b9fe9bd2749e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:55.644216 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08050621-32ee-4492-812a-b9fe9bd2749e admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:55.656557 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1305/2607] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:26:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:26:55.657469 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e2605a61-97c8-41fe-b9f5-6f9c28a18982 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc855cff70a1467d828b9d051e61588a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:55.658188 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e2605a61-97c8-41fe-b9f5-6f9c28a18982 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc855cff70a1467d828b9d051e61588a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:55.660717 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08050621-32ee-4492-812a-b9fe9bd2749e admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:55.666350 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e2605a61-97c8-41fe-b9f5-6f9c28a18982 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc855cff70a1467d828b9d051e61588a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:55.672151 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08050621-32ee-4492-812a-b9fe9bd2749e admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:55.676926 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15c4a7c-4d37-46a2-a6f3-f24219cf74ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:55.677609 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c4a7c-4d37-46a2-a6f3-f24219cf74ac admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:55.677896 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c4a7c-4d37-46a2-a6f3-f24219cf74ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:55.678178 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c4a7c-4d37-46a2-a6f3-f24219cf74ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:55.678867 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15c4a7c-4d37-46a2-a6f3-f24219cf74ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:55.678974 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e2605a61-97c8-41fe-b9f5-6f9c28a18982 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc855cff70a1467d828b9d051e61588a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:55.679710 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e2605a61-97c8-41fe-b9f5-6f9c28a18982 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc855cff70a1467d828b9d051e61588a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:55.680764 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15c4a7c-4d37-46a2-a6f3-f24219cf74ac admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:55.692848 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1303/2608] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:26:55 2022] DELETE /identity/v3/users/dc855cff70a1467d828b9d051e61588a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:26:55.711420 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1306/2609] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:26:55 2022] POST /identity/v3/projects => generated 391 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:26:55.712580 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2605a61-97c8-41fe-b9f5-6f9c28a18982 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:55.714072 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e2605a61-97c8-41fe-b9f5-6f9c28a18982 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:55.795776 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2605a61-97c8-41fe-b9f5-6f9c28a18982 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:55.862373 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:55.878923 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d1335e-dd30-4022-8ed4-e48553f1e8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:55.881031 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1335e-dd30-4022-8ed4-e48553f1e8bd admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:55.881031 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1335e-dd30-4022-8ed4-e48553f1e8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:55.881031 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1335e-dd30-4022-8ed4-e48553f1e8bd admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:55.884062 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d1335e-dd30-4022-8ed4-e48553f1e8bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:55.886457 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d1335e-dd30-4022-8ed4-e48553f1e8bd admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:55.914173 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1304/2610] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:26:55 2022] POST /identity/v3/users => generated 296 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:26:55.916770 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15c4a7c-4d37-46a2-a6f3-f24219cf74ac admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:55.933847 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15c4a7c-4d37-46a2-a6f3-f24219cf74ac admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:55.941617 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc39a87-ca5f-4ea5-a9ca-d7500d548f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:55.942428 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc39a87-ca5f-4ea5-a9ca-d7500d548f10 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:55.942785 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc39a87-ca5f-4ea5-a9ca-d7500d548f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:55.943264 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc39a87-ca5f-4ea5-a9ca-d7500d548f10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:55.943972 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc39a87-ca5f-4ea5-a9ca-d7500d548f10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:55.945712 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc39a87-ca5f-4ea5-a9ca-d7500d548f10 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:55.962187 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d1335e-dd30-4022-8ed4-e48553f1e8bd admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:55.963126 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1307/2611] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:26:55 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:26:55.974927 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d1335e-dd30-4022-8ed4-e48553f1e8bd admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:55.979695 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a62f8d-ca26-452f-9159-84749d3edac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:55.980374 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a62f8d-ca26-452f-9159-84749d3edac5 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:55.980660 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a62f8d-ca26-452f-9159-84749d3edac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:55.980937 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a62f8d-ca26-452f-9159-84749d3edac5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:55.981778 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a62f8d-ca26-452f-9159-84749d3edac5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:55.983375 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a62f8d-ca26-452f-9159-84749d3edac5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:55.995393 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1305/2612] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:26:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:26:55.999563 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc39a87-ca5f-4ea5-a9ca-d7500d548f10 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:56.010930 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc39a87-ca5f-4ea5-a9ca-d7500d548f10 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:56.016068 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc42145-3134-4101-a242-5e1d52e2b7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:56.017369 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc42145-3134-4101-a242-5e1d52e2b7d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:56.017913 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc42145-3134-4101-a242-5e1d52e2b7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:56.018228 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc42145-3134-4101-a242-5e1d52e2b7d9 admin admin] PATH_INFO: `/v3/projects/d97f48dbaf9f4f0cbff1f22093b5746c/users/e8f16f9e9c7446b3b622d206b4545ea8/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:56.031240 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a62f8d-ca26-452f-9159-84749d3edac5 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:56.035395 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-25a62f8d-ca26-452f-9159-84749d3edac5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:56.043382 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc42145-3134-4101-a242-5e1d52e2b7d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d97f48dbaf9f4f0cbff1f22093b5746c, user_id=e8f16f9e9c7446b3b622d206b4545ea8, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:56.046347 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc42145-3134-4101-a242-5e1d52e2b7d9 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:56.054809 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a62f8d-ca26-452f-9159-84749d3edac5 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:56.065375 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db352f-ce78-4c46-94e9-0ae14ac609f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:56.067038 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db352f-ce78-4c46-94e9-0ae14ac609f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:56.067618 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db352f-ce78-4c46-94e9-0ae14ac609f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:56.067998 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db352f-ce78-4c46-94e9-0ae14ac609f8 admin admin] PATH_INFO: `/v3/projects/a4e985abf5714d8cb0552c97c1a22db7` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:56.069087 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1308/2613] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:26:55 2022] PUT /identity/v3/projects/d97f48dbaf9f4f0cbff1f22093b5746c/users/e8f16f9e9c7446b3b622d206b4545ea8/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:56.069673 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94db352f-ce78-4c46-94e9-0ae14ac609f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4e985abf5714d8cb0552c97c1a22db7)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:56.071906 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94db352f-ce78-4c46-94e9-0ae14ac609f8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:56.080544 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292847d5-e0a1-4778-b368-ce6fb260a9ab None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:56.080890 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292847d5-e0a1-4778-b368-ce6fb260a9ab None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:56.081285 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292847d5-e0a1-4778-b368-ce6fb260a9ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:56.118881 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-292847d5-e0a1-4778-b368-ce6fb260a9ab None None] MFA Rules not processed for user `e8f16f9e9c7446b3b622d206b4545ea8`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:56.120216 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-292847d5-e0a1-4778-b368-ce6fb260a9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:56.150666 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:56.154044 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-94db352f-ce78-4c46-94e9-0ae14ac609f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4e985abf5714d8cb0552c97c1a22db7', 'request_id': 'req-94db352f-ce78-4c46-94e9-0ae14ac609f8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:56.162877 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-94db352f-ce78-4c46-94e9-0ae14ac609f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4e985abf5714d8cb0552c97c1a22db7', 'request_id': 'req-94db352f-ce78-4c46-94e9-0ae14ac609f8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:56.171085 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-94db352f-ce78-4c46-94e9-0ae14ac609f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4e985abf5714d8cb0552c97c1a22db7', 'request_id': 'req-94db352f-ce78-4c46-94e9-0ae14ac609f8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:56.187076 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-94db352f-ce78-4c46-94e9-0ae14ac609f8 admin admin] The token cache is being invalidate because project a4e985abf5714d8cb0552c97c1a22db7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:26:56.187474 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-94db352f-ce78-4c46-94e9-0ae14ac609f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:56.188316 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-94db352f-ce78-4c46-94e9-0ae14ac609f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:56.190293 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1306/2614] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:26:56 2022] DELETE /identity/v3/projects/a4e985abf5714d8cb0552c97c1a22db7 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:56.197513 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde761eb-ad8a-4daa-b753-22baf8faacd0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:56.198044 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde761eb-ad8a-4daa-b753-22baf8faacd0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:56.199679 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde761eb-ad8a-4daa-b753-22baf8faacd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:56.206748 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1309/2615] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:26:56.230382 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292847d5-e0a1-4778-b368-ce6fb260a9ab None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:56.231796 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-292847d5-e0a1-4778-b368-ce6fb260a9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:56.241816 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-fde761eb-ad8a-4daa-b753-22baf8faacd0 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:56.243044 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fde761eb-ad8a-4daa-b753-22baf8faacd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:56.310937 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1307/2616] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:26:56.310937 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde761eb-ad8a-4daa-b753-22baf8faacd0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:56.310937 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fde761eb-ad8a-4daa-b753-22baf8faacd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:56.311274 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292847d5-e0a1-4778-b368-ce6fb260a9ab None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:56.314596 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886a4740-3ebc-453d-9982-5bafa02c64af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:56.315464 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a4740-3ebc-453d-9982-5bafa02c64af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:56.315744 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a4740-3ebc-453d-9982-5bafa02c64af admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:56.316018 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a4740-3ebc-453d-9982-5bafa02c64af admin admin] PATH_INFO: `/v3/users/eee5f003c6274bd6b27e8de6f262491c` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:56.317246 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886a4740-3ebc-453d-9982-5bafa02c64af admin admin] RBAC: Authorizing `identity:delete_user(user_id=eee5f003c6274bd6b27e8de6f262491c)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:56.319861 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886a4740-3ebc-453d-9982-5bafa02c64af admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:56.364969 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde761eb-ad8a-4daa-b753-22baf8faacd0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:56.369018 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-886a4740-3ebc-453d-9982-5bafa02c64af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eee5f003c6274bd6b27e8de6f262491c'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:56.369260 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da439bf-6b0d-4107-bd9e-e08ce4eee646 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:56.369928 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da439bf-6b0d-4107-bd9e-e08ce4eee646 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:56.370217 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da439bf-6b0d-4107-bd9e-e08ce4eee646 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:56.370590 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da439bf-6b0d-4107-bd9e-e08ce4eee646 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:56.372332 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9da439bf-6b0d-4107-bd9e-e08ce4eee646 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:56.376945 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-886a4740-3ebc-453d-9982-5bafa02c64af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eee5f003c6274bd6b27e8de6f262491c'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:56.377670 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-886a4740-3ebc-453d-9982-5bafa02c64af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eee5f003c6274bd6b27e8de6f262491c'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:56.379303 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:56.385573 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-886a4740-3ebc-453d-9982-5bafa02c64af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eee5f003c6274bd6b27e8de6f262491c'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:56.399337 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-886a4740-3ebc-453d-9982-5bafa02c64af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eee5f003c6274bd6b27e8de6f262491c'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:56.405617 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1310/2617] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:26:56 2022] DELETE /identity/v3/users/eee5f003c6274bd6b27e8de6f262491c => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:56.407012 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e8545-68d0-4b1d-8d8a-21881090e46b None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:56.407369 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e8545-68d0-4b1d-8d8a-21881090e46b None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:56.407659 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e8545-68d0-4b1d-8d8a-21881090e46b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:56.427977 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-4b7e8545-68d0-4b1d-8d8a-21881090e46b None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:56.429223 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4b7e8545-68d0-4b1d-8d8a-21881090e46b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:56.440465 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da439bf-6b0d-4107-bd9e-e08ce4eee646 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:56.441952 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da439bf-6b0d-4107-bd9e-e08ce4eee646 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:56.452177 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1308/2618] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:26:56 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:26:56.453929 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c743e9-16b8-440d-ad62-747566e94005 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:56.454227 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c743e9-16b8-440d-ad62-747566e94005 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:56.454514 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c743e9-16b8-440d-ad62-747566e94005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:56.474523 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b8c743e9-16b8-440d-ad62-747566e94005 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:56.475872 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b8c743e9-16b8-440d-ad62-747566e94005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:56.478867 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:56.501707 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1311/2619] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:26:56.506741 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7e8545-68d0-4b1d-8d8a-21881090e46b None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:56.516993 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7e8545-68d0-4b1d-8d8a-21881090e46b None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:56.522738 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc9c58-245b-4e9a-b2e2-b4cd62a33095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:56.523649 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc9c58-245b-4e9a-b2e2-b4cd62a33095 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:56.523999 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc9c58-245b-4e9a-b2e2-b4cd62a33095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:56.524346 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc9c58-245b-4e9a-b2e2-b4cd62a33095 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:56.525177 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bc9c58-245b-4e9a-b2e2-b4cd62a33095 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:26:56.525556 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bc9c58-245b-4e9a-b2e2-b4cd62a33095 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:56.527229 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bc9c58-245b-4e9a-b2e2-b4cd62a33095 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:56.540682 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1312/2620] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:26:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:26:56.543511 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1309/2621] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:26:56.545421 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc9c58-245b-4e9a-b2e2-b4cd62a33095 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:56.556360 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c743e9-16b8-440d-ad62-747566e94005 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:56.556672 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc9c58-245b-4e9a-b2e2-b4cd62a33095 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:56.557840 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b8c743e9-16b8-440d-ad62-747566e94005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:56.561545 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb65ec1-a9eb-4db1-a64a-9b396550266a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:56.562231 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb65ec1-a9eb-4db1-a64a-9b396550266a admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:56.562527 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb65ec1-a9eb-4db1-a64a-9b396550266a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:56.562933 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb65ec1-a9eb-4db1-a64a-9b396550266a admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:56.563647 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb65ec1-a9eb-4db1-a64a-9b396550266a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:56.565857 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb65ec1-a9eb-4db1-a64a-9b396550266a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:56.600773 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1313/2622] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:26:56 2022] POST /identity/v3/projects => generated 381 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:26:56.601844 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb65ec1-a9eb-4db1-a64a-9b396550266a admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:56.625423 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c743e9-16b8-440d-ad62-747566e94005 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:56.681503 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb65ec1-a9eb-4db1-a64a-9b396550266a admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:56.686438 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52d5b8e-7e7a-416c-91bd-4fe010273ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:56.687258 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d5b8e-7e7a-416c-91bd-4fe010273ab7 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:56.687598 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d5b8e-7e7a-416c-91bd-4fe010273ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:56.688131 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d5b8e-7e7a-416c-91bd-4fe010273ab7 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:56.688840 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52d5b8e-7e7a-416c-91bd-4fe010273ab7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:56.689307 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd0f1e-49d5-4960-ae3e-e0d56f55f2f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:56.690097 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd0f1e-49d5-4960-ae3e-e0d56f55f2f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:56.690410 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd0f1e-49d5-4960-ae3e-e0d56f55f2f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:56.690766 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd0f1e-49d5-4960-ae3e-e0d56f55f2f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:56.691100 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52d5b8e-7e7a-416c-91bd-4fe010273ab7 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:56.701581 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bd0f1e-49d5-4960-ae3e-e0d56f55f2f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:56.703370 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bd0f1e-49d5-4960-ae3e-e0d56f55f2f2 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:56.715392 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1314/2623] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:26:56 2022] POST /identity/v3/users => generated 291 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:26:56.716803 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1310/2624] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:26:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:26:56.720232 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52d5b8e-7e7a-416c-91bd-4fe010273ab7 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:56.732865 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52d5b8e-7e7a-416c-91bd-4fe010273ab7 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:56.737952 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6237bc23-215a-4480-a326-502178b519e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:56.738778 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6237bc23-215a-4480-a326-502178b519e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:56.739127 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6237bc23-215a-4480-a326-502178b519e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:56.739648 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6237bc23-215a-4480-a326-502178b519e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:56.740557 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6237bc23-215a-4480-a326-502178b519e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:56.742539 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6237bc23-215a-4480-a326-502178b519e5 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:56.769422 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1315/2625] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:26:56.773116 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd0f1e-49d5-4960-ae3e-e0d56f55f2f2 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:56.790564 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd0f1e-49d5-4960-ae3e-e0d56f55f2f2 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:56.799101 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1587c6-53de-4412-b546-6278b93f3281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:56.800299 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1587c6-53de-4412-b546-6278b93f3281 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:56.800930 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1587c6-53de-4412-b546-6278b93f3281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:56.801492 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1587c6-53de-4412-b546-6278b93f3281 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:56.803391 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1587c6-53de-4412-b546-6278b93f3281 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:56.807043 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1587c6-53de-4412-b546-6278b93f3281 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:56.809396 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:56.834012 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1311/2626] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:26:56.834407 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6237bc23-215a-4480-a326-502178b519e5 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:56.843884 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6237bc23-215a-4480-a326-502178b519e5 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:56.849956 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f978833c-a336-4750-8a89-2a274b9ff6dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:56.850891 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f978833c-a336-4750-8a89-2a274b9ff6dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:56.851182 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f978833c-a336-4750-8a89-2a274b9ff6dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:56.851488 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f978833c-a336-4750-8a89-2a274b9ff6dd admin admin] PATH_INFO: `/v3/projects/8b7d1873b50b4797a00b4ccfc05d6357/users/cd4f0d7d19324b8996f5daa1f14fd27f/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:56.869551 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f978833c-a336-4750-8a89-2a274b9ff6dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b7d1873b50b4797a00b4ccfc05d6357, user_id=cd4f0d7d19324b8996f5daa1f14fd27f, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:56.872279 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f978833c-a336-4750-8a89-2a274b9ff6dd admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:56.888833 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1316/2627] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:26:56 2022] PUT /identity/v3/projects/8b7d1873b50b4797a00b4ccfc05d6357/users/cd4f0d7d19324b8996f5daa1f14fd27f/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:56.899889 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e8ca7-25a4-4094-b17f-85d91fd98b50 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:56.900242 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e8ca7-25a4-4094-b17f-85d91fd98b50 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:56.900453 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e8ca7-25a4-4094-b17f-85d91fd98b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:56.930097 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-fa3e8ca7-25a4-4094-b17f-85d91fd98b50 None None] MFA Rules not processed for user `cd4f0d7d19324b8996f5daa1f14fd27f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:56.931332 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fa3e8ca7-25a4-4094-b17f-85d91fd98b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:57.001363 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1312/2628] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:26:57.005621 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f978833c-a336-4750-8a89-2a274b9ff6dd admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:57.029928 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3e8ca7-25a4-4094-b17f-85d91fd98b50 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:57.031557 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fa3e8ca7-25a4-4094-b17f-85d91fd98b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:57.078757 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f978833c-a336-4750-8a89-2a274b9ff6dd admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:57.085674 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0e9fc9-2910-4574-a49f-6a1c152a27fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:57.086554 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e9fc9-2910-4574-a49f-6a1c152a27fa admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:57.086841 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e9fc9-2910-4574-a49f-6a1c152a27fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:57.087118 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e9fc9-2910-4574-a49f-6a1c152a27fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:57.087935 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0e9fc9-2910-4574-a49f-6a1c152a27fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:57.090146 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0e9fc9-2910-4574-a49f-6a1c152a27fa admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:57.100612 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:57.106953 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3e8ca7-25a4-4094-b17f-85d91fd98b50 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:57.111876 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1cd4b3-beea-4f90-a8c2-27f245ad9dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:57.112837 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1cd4b3-beea-4f90-a8c2-27f245ad9dc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:57.113156 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1cd4b3-beea-4f90-a8c2-27f245ad9dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:57.113441 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1cd4b3-beea-4f90-a8c2-27f245ad9dc2 admin admin] PATH_INFO: `/v3/projects/888e470954834c2796e4d1872fa14e5c` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:57.114745 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1cd4b3-beea-4f90-a8c2-27f245ad9dc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=888e470954834c2796e4d1872fa14e5c)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:57.116679 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1cd4b3-beea-4f90-a8c2-27f245ad9dc2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:57.121083 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1317/2629] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:26:57 2022] POST /identity/v3/projects => generated 383 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:26:57.124774 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0e9fc9-2910-4574-a49f-6a1c152a27fa admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:57.165867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8b1cd4b3-beea-4f90-a8c2-27f245ad9dc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '888e470954834c2796e4d1872fa14e5c', 'request_id': 'req-8b1cd4b3-beea-4f90-a8c2-27f245ad9dc2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:57.171827 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8b1cd4b3-beea-4f90-a8c2-27f245ad9dc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '888e470954834c2796e4d1872fa14e5c', 'request_id': 'req-8b1cd4b3-beea-4f90-a8c2-27f245ad9dc2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:57.178261 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8b1cd4b3-beea-4f90-a8c2-27f245ad9dc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '888e470954834c2796e4d1872fa14e5c', 'request_id': 'req-8b1cd4b3-beea-4f90-a8c2-27f245ad9dc2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:57.191750 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0e9fc9-2910-4574-a49f-6a1c152a27fa admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:57.191996 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8b1cd4b3-beea-4f90-a8c2-27f245ad9dc2 admin admin] The token cache is being invalidate because project 888e470954834c2796e4d1872fa14e5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:26:57.192816 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8b1cd4b3-beea-4f90-a8c2-27f245ad9dc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:57.193485 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8b1cd4b3-beea-4f90-a8c2-27f245ad9dc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:57.195072 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1313/2630] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:26:57 2022] DELETE /identity/v3/projects/888e470954834c2796e4d1872fa14e5c => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:26:57.198798 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1cd4b3-beea-4f90-a8c2-27f245ad9dc2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:57.200107 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8b1cd4b3-beea-4f90-a8c2-27f245ad9dc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:57.220316 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:57.258203 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5566825-fa3d-48a1-a9dd-c7daab11e258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:57.258889 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5566825-fa3d-48a1-a9dd-c7daab11e258 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:57.259181 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5566825-fa3d-48a1-a9dd-c7daab11e258 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:57.259623 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5566825-fa3d-48a1-a9dd-c7daab11e258 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:57.260160 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5566825-fa3d-48a1-a9dd-c7daab11e258 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:57.262157 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5566825-fa3d-48a1-a9dd-c7daab11e258 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:57.276688 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1cd4b3-beea-4f90-a8c2-27f245ad9dc2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:57.282573 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd96e616-3c56-48d6-a41e-264aedc0513f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:57.283484 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96e616-3c56-48d6-a41e-264aedc0513f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:57.283767 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96e616-3c56-48d6-a41e-264aedc0513f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:57.284191 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96e616-3c56-48d6-a41e-264aedc0513f admin admin] PATH_INFO: `/v3/users/09230860bc9e48c0a6bf9106551fa063` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:57.285387 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1318/2631] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:26:57 2022] POST /identity/v3/users => generated 298 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:26:57.285512 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd96e616-3c56-48d6-a41e-264aedc0513f admin admin] RBAC: Authorizing `identity:delete_user(user_id=09230860bc9e48c0a6bf9106551fa063)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:57.288343 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd96e616-3c56-48d6-a41e-264aedc0513f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:57.289424 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5566825-fa3d-48a1-a9dd-c7daab11e258 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:57.291124 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f5566825-fa3d-48a1-a9dd-c7daab11e258 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:57.303891 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5566825-fa3d-48a1-a9dd-c7daab11e258 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:57.308635 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389e42b0-8883-4891-9c60-c96b2dae9177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:57.309355 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389e42b0-8883-4891-9c60-c96b2dae9177 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:57.309661 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389e42b0-8883-4891-9c60-c96b2dae9177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:57.309934 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389e42b0-8883-4891-9c60-c96b2dae9177 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:57.310650 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389e42b0-8883-4891-9c60-c96b2dae9177 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:57.312235 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389e42b0-8883-4891-9c60-c96b2dae9177 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:57.323785 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1319/2632] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:26:57.328234 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389e42b0-8883-4891-9c60-c96b2dae9177 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:57.338590 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389e42b0-8883-4891-9c60-c96b2dae9177 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:57.341431 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dd96e616-3c56-48d6-a41e-264aedc0513f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09230860bc9e48c0a6bf9106551fa063'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:57.342104 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dd96e616-3c56-48d6-a41e-264aedc0513f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09230860bc9e48c0a6bf9106551fa063'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:57.343213 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d7dc78-d043-4eef-b28c-c5f26f530a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:57.344332 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7dc78-d043-4eef-b28c-c5f26f530a29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:57.344618 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7dc78-d043-4eef-b28c-c5f26f530a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:57.344892 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7dc78-d043-4eef-b28c-c5f26f530a29 admin admin] PATH_INFO: `/v3/projects/ec1e84e6e2d542e5bef64738b90e63d2/users/563b36b0f203439ba623ca7e9cc3a212/roles/f36e053b442540f08041ac7878760f65` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:57.350053 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dd96e616-3c56-48d6-a41e-264aedc0513f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09230860bc9e48c0a6bf9106551fa063'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:57.360952 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d7dc78-d043-4eef-b28c-c5f26f530a29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec1e84e6e2d542e5bef64738b90e63d2, user_id=563b36b0f203439ba623ca7e9cc3a212, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:57.362411 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dd96e616-3c56-48d6-a41e-264aedc0513f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09230860bc9e48c0a6bf9106551fa063'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:57.363077 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dd96e616-3c56-48d6-a41e-264aedc0513f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09230860bc9e48c0a6bf9106551fa063'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:57.363648 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d7dc78-d043-4eef-b28c-c5f26f530a29 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:57.376040 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1314/2633] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:26:57 2022] DELETE /identity/v3/users/09230860bc9e48c0a6bf9106551fa063 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:26:57.379165 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1320/2634] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:26:57 2022] PUT /identity/v3/projects/ec1e84e6e2d542e5bef64738b90e63d2/users/563b36b0f203439ba623ca7e9cc3a212/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:26:57.381993 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd96e616-3c56-48d6-a41e-264aedc0513f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:57.383368 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dd96e616-3c56-48d6-a41e-264aedc0513f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:57.452037 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd96e616-3c56-48d6-a41e-264aedc0513f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:57.458817 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6046d1d8-e87e-4ac3-98e6-8d7f91ff2f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:57.459496 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6046d1d8-e87e-4ac3-98e6-8d7f91ff2f5f admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:57.459779 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6046d1d8-e87e-4ac3-98e6-8d7f91ff2f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:57.460057 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6046d1d8-e87e-4ac3-98e6-8d7f91ff2f5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:57.460781 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6046d1d8-e87e-4ac3-98e6-8d7f91ff2f5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:57.462400 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6046d1d8-e87e-4ac3-98e6-8d7f91ff2f5f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:57.473202 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1315/2635] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:26:57.477361 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d7dc78-d043-4eef-b28c-c5f26f530a29 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:57.535523 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:57.550357 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d7dc78-d043-4eef-b28c-c5f26f530a29 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:57.556749 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c80bba0-f6f6-4779-984f-83f09700000f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:57.557673 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c80bba0-f6f6-4779-984f-83f09700000f admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:57.557957 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c80bba0-f6f6-4779-984f-83f09700000f admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:57.558230 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c80bba0-f6f6-4779-984f-83f09700000f admin admin] PATH_INFO: `/v3/projects/ec1e84e6e2d542e5bef64738b90e63d2/users/563b36b0f203439ba623ca7e9cc3a212/roles/f36e053b442540f08041ac7878760f65` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:57.560200 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c80bba0-f6f6-4779-984f-83f09700000f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec1e84e6e2d542e5bef64738b90e63d2, user_id=563b36b0f203439ba623ca7e9cc3a212, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:57.562907 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c80bba0-f6f6-4779-984f-83f09700000f admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:57.575396 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1321/2636] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:26:57 2022] PUT /identity/v3/projects/ec1e84e6e2d542e5bef64738b90e63d2/users/563b36b0f203439ba623ca7e9cc3a212/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:57.642020 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19200d-b6fa-417b-be7f-29c50894bec1 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:57.642341 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19200d-b6fa-417b-be7f-29c50894bec1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:57.642621 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19200d-b6fa-417b-be7f-29c50894bec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:57.670340 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c80bba0-f6f6-4779-984f-83f09700000f admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:57.672526 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-5c80bba0-f6f6-4779-984f-83f09700000f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:57.673718 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9a19200d-b6fa-417b-be7f-29c50894bec1 None None] MFA Rules not processed for user `563b36b0f203439ba623ca7e9cc3a212`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:57.674877 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9a19200d-b6fa-417b-be7f-29c50894bec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:57.710339 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:57.746469 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c80bba0-f6f6-4779-984f-83f09700000f admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:57.752413 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fe41a3-7606-4b41-814e-0be473055cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:57.753626 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe41a3-7606-4b41-814e-0be473055cb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:57.753907 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe41a3-7606-4b41-814e-0be473055cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:57.754179 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe41a3-7606-4b41-814e-0be473055cb9 admin admin] PATH_INFO: `/v3/projects/02b8a4dafba54c229d0650e915f774d2` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:57.756337 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fe41a3-7606-4b41-814e-0be473055cb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02b8a4dafba54c229d0650e915f774d2)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:57.758854 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fe41a3-7606-4b41-814e-0be473055cb9 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:57.766049 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1316/2637] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:57 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:26:57.778222 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19200d-b6fa-417b-be7f-29c50894bec1 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:57.779530 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9a19200d-b6fa-417b-be7f-29c50894bec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:57.810976 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-88fe41a3-7606-4b41-814e-0be473055cb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02b8a4dafba54c229d0650e915f774d2', 'request_id': 'req-88fe41a3-7606-4b41-814e-0be473055cb9'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:57.816501 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-88fe41a3-7606-4b41-814e-0be473055cb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02b8a4dafba54c229d0650e915f774d2', 'request_id': 'req-88fe41a3-7606-4b41-814e-0be473055cb9'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:57.824233 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-88fe41a3-7606-4b41-814e-0be473055cb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02b8a4dafba54c229d0650e915f774d2', 'request_id': 'req-88fe41a3-7606-4b41-814e-0be473055cb9'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:57.840655 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-88fe41a3-7606-4b41-814e-0be473055cb9 admin admin] The token cache is being invalidate because project 02b8a4dafba54c229d0650e915f774d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:26:57.841006 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-88fe41a3-7606-4b41-814e-0be473055cb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:57.841741 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-88fe41a3-7606-4b41-814e-0be473055cb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:57.843503 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1322/2638] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:26:57 2022] DELETE /identity/v3/projects/02b8a4dafba54c229d0650e915f774d2 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:26:57.845184 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19200d-b6fa-417b-be7f-29c50894bec1 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:57.848972 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7616a8c-3ff3-4783-80fd-f4024c6ae394 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:57.849470 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7616a8c-3ff3-4783-80fd-f4024c6ae394 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:57.849755 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7616a8c-3ff3-4783-80fd-f4024c6ae394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:57.870746 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-d7616a8c-3ff3-4783-80fd-f4024c6ae394 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:57.872063 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d7616a8c-3ff3-4783-80fd-f4024c6ae394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:57.906243 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27d9033-7e68-4c5c-ab2e-1ec112e3b082 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:57.906903 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d9033-7e68-4c5c-ab2e-1ec112e3b082 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:57.907186 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d9033-7e68-4c5c-ab2e-1ec112e3b082 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:57.907463 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d9033-7e68-4c5c-ab2e-1ec112e3b082 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:57.909514 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a27d9033-7e68-4c5c-ab2e-1ec112e3b082 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:57.943641 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1323/2639] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:26:57.951375 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e18d402-6dd6-418a-95f5-6051ba7de47f None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:57.951821 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e18d402-6dd6-418a-95f5-6051ba7de47f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:57.952117 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e18d402-6dd6-418a-95f5-6051ba7de47f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:57.976553 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-8e18d402-6dd6-418a-95f5-6051ba7de47f None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:57.977874 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-8e18d402-6dd6-418a-95f5-6051ba7de47f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:57.979331 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27d9033-7e68-4c5c-ab2e-1ec112e3b082 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:57.980786 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27d9033-7e68-4c5c-ab2e-1ec112e3b082 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:57.989699 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1324/2640] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:26:57.991268 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1317/2641] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:26:57 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:26:57.994261 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e18d402-6dd6-418a-95f5-6051ba7de47f None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:58.006062 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e18d402-6dd6-418a-95f5-6051ba7de47f None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:58.011289 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cb5a53-0aec-4d10-ae5d-bb9fc74a1266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:58.011933 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb5a53-0aec-4d10-ae5d-bb9fc74a1266 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:58.012221 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb5a53-0aec-4d10-ae5d-bb9fc74a1266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:58.012504 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb5a53-0aec-4d10-ae5d-bb9fc74a1266 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:58.013286 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cb5a53-0aec-4d10-ae5d-bb9fc74a1266 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:26:58.013602 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cb5a53-0aec-4d10-ae5d-bb9fc74a1266 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:58.015213 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cb5a53-0aec-4d10-ae5d-bb9fc74a1266 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:58.018224 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:58.040402 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27d9033-7e68-4c5c-ab2e-1ec112e3b082 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:58.045255 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1325/2642] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:26:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:26:58.054302 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27d9033-7e68-4c5c-ab2e-1ec112e3b082 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:58.062283 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a9627a-4ac5-4389-bbab-c2a5b6426832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:58.062962 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a9627a-4ac5-4389-bbab-c2a5b6426832 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:58.063250 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a9627a-4ac5-4389-bbab-c2a5b6426832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:58.063529 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a9627a-4ac5-4389-bbab-c2a5b6426832 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:58.064247 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a9627a-4ac5-4389-bbab-c2a5b6426832 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:58.066521 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a9627a-4ac5-4389-bbab-c2a5b6426832 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:58.091883 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cb5a53-0aec-4d10-ae5d-bb9fc74a1266 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:58.094545 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-69cb5a53-0aec-4d10-ae5d-bb9fc74a1266 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:58.098207 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:58.110359 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1318/2643] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:26:58 2022] POST /identity/v3/projects => generated 385 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:26:58.110528 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cb5a53-0aec-4d10-ae5d-bb9fc74a1266 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:58.111373 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a9627a-4ac5-4389-bbab-c2a5b6426832 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:58.159315 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd1c0e-45f1-482c-90a0-b68ed684634f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:58.159968 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd1c0e-45f1-482c-90a0-b68ed684634f service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:58.160255 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd1c0e-45f1-482c-90a0-b68ed684634f service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:58.160535 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd1c0e-45f1-482c-90a0-b68ed684634f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:58.162964 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-27bd1c0e-45f1-482c-90a0-b68ed684634f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:58.179432 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a9627a-4ac5-4389-bbab-c2a5b6426832 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:58.184743 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1256d439-9743-460d-a14c-498a37430581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:58.185638 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1256d439-9743-460d-a14c-498a37430581 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:58.186003 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1256d439-9743-460d-a14c-498a37430581 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:58.186360 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1256d439-9743-460d-a14c-498a37430581 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:58.187266 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1256d439-9743-460d-a14c-498a37430581 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:58.189364 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1256d439-9743-460d-a14c-498a37430581 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:58.210376 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1319/2644] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:26:58 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:26:58.215163 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1256d439-9743-460d-a14c-498a37430581 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:58.226736 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1256d439-9743-460d-a14c-498a37430581 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:58.230830 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bd1c0e-45f1-482c-90a0-b68ed684634f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:58.231424 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a01ce4d-7f63-4429-9c07-bef39e5ac717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:58.232083 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a01ce4d-7f63-4429-9c07-bef39e5ac717 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:58.232271 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bd1c0e-45f1-482c-90a0-b68ed684634f service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:58.232394 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a01ce4d-7f63-4429-9c07-bef39e5ac717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:58.232633 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a01ce4d-7f63-4429-9c07-bef39e5ac717 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:58.233389 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a01ce4d-7f63-4429-9c07-bef39e5ac717 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:58.234966 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a01ce4d-7f63-4429-9c07-bef39e5ac717 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:58.241917 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1326/2645] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:26:58 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:26:58.245327 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1320/2646] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:26:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:26:58.249293 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd1c0e-45f1-482c-90a0-b68ed684634f service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:58.260032 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd1c0e-45f1-482c-90a0-b68ed684634f service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:58.268881 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6559775d-5332-457f-b227-486a8620a905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:58.269521 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559775d-5332-457f-b227-486a8620a905 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:58.269805 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559775d-5332-457f-b227-486a8620a905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:58.270147 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559775d-5332-457f-b227-486a8620a905 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:58.272014 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6559775d-5332-457f-b227-486a8620a905 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:58.273632 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6559775d-5332-457f-b227-486a8620a905 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:58.288750 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1327/2647] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:26:58 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:26:58.292857 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a01ce4d-7f63-4429-9c07-bef39e5ac717 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:58.306017 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a01ce4d-7f63-4429-9c07-bef39e5ac717 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:58.313946 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a95e7a-b428-45e8-961e-479317060a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:58.314845 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a95e7a-b428-45e8-961e-479317060a85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:58.315133 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a95e7a-b428-45e8-961e-479317060a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:58.315540 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a95e7a-b428-45e8-961e-479317060a85 admin admin] PATH_INFO: `/v3/projects/a482a5dd6f014193a66ffae9047b0b6a/users/da129bb8259948418cd3b9b6d1c89921/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:58.335303 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a95e7a-b428-45e8-961e-479317060a85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a482a5dd6f014193a66ffae9047b0b6a, user_id=da129bb8259948418cd3b9b6d1c89921, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:58.338147 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a95e7a-b428-45e8-961e-479317060a85 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:58.369299 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1321/2648] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:26:58 2022] PUT /identity/v3/projects/a482a5dd6f014193a66ffae9047b0b6a/users/da129bb8259948418cd3b9b6d1c89921/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:58.380285 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6d2911-7050-47de-8d8b-1e21b7c27811 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:58.380884 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6d2911-7050-47de-8d8b-1e21b7c27811 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:58.380884 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6d2911-7050-47de-8d8b-1e21b7c27811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:58.410433 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-cd6d2911-7050-47de-8d8b-1e21b7c27811 None None] MFA Rules not processed for user `da129bb8259948418cd3b9b6d1c89921`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:58.412308 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-cd6d2911-7050-47de-8d8b-1e21b7c27811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:58.489121 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1328/2649] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:26:58.492648 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a95e7a-b428-45e8-961e-479317060a85 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:58.512160 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6d2911-7050-47de-8d8b-1e21b7c27811 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:58.513663 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-cd6d2911-7050-47de-8d8b-1e21b7c27811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:58.547473 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:58.560303 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a95e7a-b428-45e8-961e-479317060a85 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:58.565522 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72311942-e3c0-4102-891d-e07185537e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:58.566187 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72311942-e3c0-4102-891d-e07185537e9c admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:58.566476 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72311942-e3c0-4102-891d-e07185537e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:58.566752 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72311942-e3c0-4102-891d-e07185537e9c admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:58.567970 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72311942-e3c0-4102-891d-e07185537e9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:58.569908 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72311942-e3c0-4102-891d-e07185537e9c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:58.582035 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6d2911-7050-47de-8d8b-1e21b7c27811 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:58.588278 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f619d301-fa9a-4da4-918b-8eb18568c869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:58.589638 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619d301-fa9a-4da4-918b-8eb18568c869 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:58.589944 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619d301-fa9a-4da4-918b-8eb18568c869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:58.590230 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619d301-fa9a-4da4-918b-8eb18568c869 admin admin] PATH_INFO: `/v3/users/cd4f0d7d19324b8996f5daa1f14fd27f` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:58.592767 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f619d301-fa9a-4da4-918b-8eb18568c869 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd4f0d7d19324b8996f5daa1f14fd27f)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:58.595060 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f619d301-fa9a-4da4-918b-8eb18568c869 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:58.596985 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1322/2650] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:26:58 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:26:58.602087 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72311942-e3c0-4102-891d-e07185537e9c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:58.643061 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f619d301-fa9a-4da4-918b-8eb18568c869 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd4f0d7d19324b8996f5daa1f14fd27f'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:58.649371 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:58.651747 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f619d301-fa9a-4da4-918b-8eb18568c869 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd4f0d7d19324b8996f5daa1f14fd27f'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:58.652493 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f619d301-fa9a-4da4-918b-8eb18568c869 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd4f0d7d19324b8996f5daa1f14fd27f'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:58.659769 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f619d301-fa9a-4da4-918b-8eb18568c869 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd4f0d7d19324b8996f5daa1f14fd27f'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:58.671496 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f619d301-fa9a-4da4-918b-8eb18568c869 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd4f0d7d19324b8996f5daa1f14fd27f'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:58.671643 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72311942-e3c0-4102-891d-e07185537e9c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:58.676234 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb124c1c-3b5a-47b0-b2f2-11301ef5cd6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:58.677420 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb124c1c-3b5a-47b0-b2f2-11301ef5cd6e admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:58.677627 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1329/2651] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:26:58 2022] DELETE /identity/v3/users/cd4f0d7d19324b8996f5daa1f14fd27f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:58.680623 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb124c1c-3b5a-47b0-b2f2-11301ef5cd6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:58.680623 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb124c1c-3b5a-47b0-b2f2-11301ef5cd6e admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:58.680623 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb124c1c-3b5a-47b0-b2f2-11301ef5cd6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:58.680901 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb124c1c-3b5a-47b0-b2f2-11301ef5cd6e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:58.681981 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495c9306-e956-4735-8f98-4da2c3802e7d None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:58.682299 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495c9306-e956-4735-8f98-4da2c3802e7d None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:58.682582 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495c9306-e956-4735-8f98-4da2c3802e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:58.700349 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-495c9306-e956-4735-8f98-4da2c3802e7d None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:58.701685 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-495c9306-e956-4735-8f98-4da2c3802e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:58.702275 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1323/2652] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:26:58 2022] POST /identity/v3/users => generated 299 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:26:58.706199 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb124c1c-3b5a-47b0-b2f2-11301ef5cd6e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:58.708564 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-eb124c1c-3b5a-47b0-b2f2-11301ef5cd6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:58.721129 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb124c1c-3b5a-47b0-b2f2-11301ef5cd6e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:58.768545 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1330/2653] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:26:58.778609 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495c9306-e956-4735-8f98-4da2c3802e7d None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:58.780018 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-495c9306-e956-4735-8f98-4da2c3802e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:58.780567 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d0d414-7cf5-44ff-883b-6162b0d9363d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:58.781306 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d0d414-7cf5-44ff-883b-6162b0d9363d admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:58.781598 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d0d414-7cf5-44ff-883b-6162b0d9363d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:58.781880 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d0d414-7cf5-44ff-883b-6162b0d9363d admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:58.782610 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d0d414-7cf5-44ff-883b-6162b0d9363d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:58.784206 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d0d414-7cf5-44ff-883b-6162b0d9363d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:58.798347 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1324/2654] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:26:58 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:26:58.799125 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d0d414-7cf5-44ff-883b-6162b0d9363d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:58.809263 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d0d414-7cf5-44ff-883b-6162b0d9363d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:58.814042 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d078a084-d35b-48f6-8b19-81cab77c6012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:58.814986 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d078a084-d35b-48f6-8b19-81cab77c6012 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:58.815271 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d078a084-d35b-48f6-8b19-81cab77c6012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:58.815551 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d078a084-d35b-48f6-8b19-81cab77c6012 admin admin] PATH_INFO: `/v3/projects/d275c7fc073847f69222fca8eac554fd/users/57f9e9a9bed749cfbdd180086b4f6d56/roles/f36e053b442540f08041ac7878760f65` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:58.830781 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d078a084-d35b-48f6-8b19-81cab77c6012 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d275c7fc073847f69222fca8eac554fd, user_id=57f9e9a9bed749cfbdd180086b4f6d56, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:58.834161 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d078a084-d35b-48f6-8b19-81cab77c6012 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:58.836321 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:58.841844 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495c9306-e956-4735-8f98-4da2c3802e7d None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:58.848969 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1325/2655] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:26:58 2022] PUT /identity/v3/projects/d275c7fc073847f69222fca8eac554fd/users/57f9e9a9bed749cfbdd180086b4f6d56/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:26:58.853428 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d078a084-d35b-48f6-8b19-81cab77c6012 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:58.898836 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbedb90e-bc4f-40be-9c11-60488095d004 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:58.899802 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbedb90e-bc4f-40be-9c11-60488095d004 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:58.900089 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbedb90e-bc4f-40be-9c11-60488095d004 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:58.900366 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbedb90e-bc4f-40be-9c11-60488095d004 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:58.915575 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d078a084-d35b-48f6-8b19-81cab77c6012 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:58.920551 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e213ccf-6752-477b-851c-4812ef88352c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:58.921267 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e213ccf-6752-477b-851c-4812ef88352c admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:58.921557 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e213ccf-6752-477b-851c-4812ef88352c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:58.921835 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e213ccf-6752-477b-851c-4812ef88352c admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:58.922560 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e213ccf-6752-477b-851c-4812ef88352c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:58.924144 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e213ccf-6752-477b-851c-4812ef88352c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:58.935696 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1326/2656] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:26:58 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:26:58.939389 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e213ccf-6752-477b-851c-4812ef88352c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:58.949488 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e213ccf-6752-477b-851c-4812ef88352c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:58.954392 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa25099-590f-4a9e-9648-915a909d8f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:58.955392 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa25099-590f-4a9e-9648-915a909d8f07 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:58.955685 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa25099-590f-4a9e-9648-915a909d8f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:58.955968 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa25099-590f-4a9e-9648-915a909d8f07 admin admin] PATH_INFO: `/v3/projects/d275c7fc073847f69222fca8eac554fd/users/57f9e9a9bed749cfbdd180086b4f6d56/roles/f36e053b442540f08041ac7878760f65` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:58.958696 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa25099-590f-4a9e-9648-915a909d8f07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d275c7fc073847f69222fca8eac554fd, user_id=57f9e9a9bed749cfbdd180086b4f6d56, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:58.961401 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa25099-590f-4a9e-9648-915a909d8f07 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:58.963314 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbedb90e-bc4f-40be-9c11-60488095d004 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:58.964762 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbedb90e-bc4f-40be-9c11-60488095d004 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:58.971785 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1327/2657] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:26:58 2022] PUT /identity/v3/projects/d275c7fc073847f69222fca8eac554fd/users/57f9e9a9bed749cfbdd180086b4f6d56/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:26:58.975231 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1331/2658] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:26:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:26:58.982152 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1c080-4718-43c5-830e-2dc551fdf986 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:58.982451 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1c080-4718-43c5-830e-2dc551fdf986 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:58.982739 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1c080-4718-43c5-830e-2dc551fdf986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:59.010502 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-eeb1c080-4718-43c5-830e-2dc551fdf986 None None] MFA Rules not processed for user `57f9e9a9bed749cfbdd180086b4f6d56`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:59.011767 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-eeb1c080-4718-43c5-830e-2dc551fdf986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:59.024523 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:59.085895 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1328/2659] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:58 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:26:59.097991 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbedb90e-bc4f-40be-9c11-60488095d004 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:59.100140 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-dbedb90e-bc4f-40be-9c11-60488095d004 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:59.173900 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbedb90e-bc4f-40be-9c11-60488095d004 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:59.227631 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4442201-c701-41cb-a053-c9c2d8501020 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:59.228300 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4442201-c701-41cb-a053-c9c2d8501020 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:59.228588 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4442201-c701-41cb-a053-c9c2d8501020 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:59.228872 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4442201-c701-41cb-a053-c9c2d8501020 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:59.239483 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4442201-c701-41cb-a053-c9c2d8501020 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:59.241333 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4442201-c701-41cb-a053-c9c2d8501020 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:59.252727 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1332/2660] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:26:59 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:26:59.254194 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb1c080-4718-43c5-830e-2dc551fdf986 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:59.256503 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-eeb1c080-4718-43c5-830e-2dc551fdf986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:59.287283 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4442201-c701-41cb-a053-c9c2d8501020 service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:59.300015 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:59.337612 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb1c080-4718-43c5-830e-2dc551fdf986 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:59.342126 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4418c930-d2d8-449d-8b93-65057a3c9988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:59.342986 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418c930-d2d8-449d-8b93-65057a3c9988 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:59.343269 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418c930-d2d8-449d-8b93-65057a3c9988 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:59.343543 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418c930-d2d8-449d-8b93-65057a3c9988 admin admin] PATH_INFO: `/v3/projects/8b7d1873b50b4797a00b4ccfc05d6357` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:59.344981 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4418c930-d2d8-449d-8b93-65057a3c9988 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b7d1873b50b4797a00b4ccfc05d6357)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:59.347434 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4418c930-d2d8-449d-8b93-65057a3c9988 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:59.351535 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4442201-c701-41cb-a053-c9c2d8501020 service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:59.356414 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1bf338-41fa-495c-9941-3829a017dfc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:59.357330 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1bf338-41fa-495c-9941-3829a017dfc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:59.357841 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1bf338-41fa-495c-9941-3829a017dfc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:59.358129 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1bf338-41fa-495c-9941-3829a017dfc2 admin admin] PATH_INFO: `/v3/users/da129bb8259948418cd3b9b6d1c89921` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:59.359349 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1bf338-41fa-495c-9941-3829a017dfc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da129bb8259948418cd3b9b6d1c89921)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:59.361392 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1bf338-41fa-495c-9941-3829a017dfc2 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:59.413958 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4418c930-d2d8-449d-8b93-65057a3c9988 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b7d1873b50b4797a00b4ccfc05d6357', 'request_id': 'req-4418c930-d2d8-449d-8b93-65057a3c9988'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:59.419177 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4418c930-d2d8-449d-8b93-65057a3c9988 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b7d1873b50b4797a00b4ccfc05d6357', 'request_id': 'req-4418c930-d2d8-449d-8b93-65057a3c9988'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:59.420557 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4e1bf338-41fa-495c-9941-3829a017dfc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da129bb8259948418cd3b9b6d1c89921'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:59.425628 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4418c930-d2d8-449d-8b93-65057a3c9988 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b7d1873b50b4797a00b4ccfc05d6357', 'request_id': 'req-4418c930-d2d8-449d-8b93-65057a3c9988'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:59.428556 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4e1bf338-41fa-495c-9941-3829a017dfc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da129bb8259948418cd3b9b6d1c89921'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:59.429281 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4e1bf338-41fa-495c-9941-3829a017dfc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da129bb8259948418cd3b9b6d1c89921'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:59.436982 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4e1bf338-41fa-495c-9941-3829a017dfc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da129bb8259948418cd3b9b6d1c89921'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:59.439114 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4418c930-d2d8-449d-8b93-65057a3c9988 admin admin] The token cache is being invalidate because project 8b7d1873b50b4797a00b4ccfc05d6357 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:26:59.439478 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4418c930-d2d8-449d-8b93-65057a3c9988 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:59.440089 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4418c930-d2d8-449d-8b93-65057a3c9988 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:59.441674 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1329/2661] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:26:59 2022] DELETE /identity/v3/projects/8b7d1873b50b4797a00b4ccfc05d6357 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:26:59.446466 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4418c930-d2d8-449d-8b93-65057a3c9988 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:59.447908 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4418c930-d2d8-449d-8b93-65057a3c9988 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:59.450824 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4e1bf338-41fa-495c-9941-3829a017dfc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da129bb8259948418cd3b9b6d1c89921'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:59.458971 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1333/2662] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:26:59 2022] DELETE /identity/v3/users/da129bb8259948418cd3b9b6d1c89921 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:26:59.463451 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac772b-62b2-42c1-b18d-db32486d86a1 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:59.463825 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac772b-62b2-42c1-b18d-db32486d86a1 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:59.464291 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac772b-62b2-42c1-b18d-db32486d86a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:59.486139 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-2eac772b-62b2-42c1-b18d-db32486d86a1 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:59.487380 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-2eac772b-62b2-42c1-b18d-db32486d86a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:59.497662 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:59.524848 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4418c930-d2d8-449d-8b93-65057a3c9988 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:59.549661 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1334/2663] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:26:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:26:59.559577 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eac772b-62b2-42c1-b18d-db32486d86a1 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:59.561001 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-2eac772b-62b2-42c1-b18d-db32486d86a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:59.581666 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa3fd67-a508-43b1-b74e-ec0917b965d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:59.582659 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa3fd67-a508-43b1-b74e-ec0917b965d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:59.583905 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa3fd67-a508-43b1-b74e-ec0917b965d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:59.583905 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa3fd67-a508-43b1-b74e-ec0917b965d9 admin admin] PATH_INFO: `/v3/users/563b36b0f203439ba623ca7e9cc3a212` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:59.585189 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa3fd67-a508-43b1-b74e-ec0917b965d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=563b36b0f203439ba623ca7e9cc3a212)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:59.587098 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa3fd67-a508-43b1-b74e-ec0917b965d9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:59.597892 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:59.622835 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eac772b-62b2-42c1-b18d-db32486d86a1 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:59.626545 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b062763-08bb-4ed7-8405-0396b5ec248e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:59.627202 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b062763-08bb-4ed7-8405-0396b5ec248e service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:59.627486 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b062763-08bb-4ed7-8405-0396b5ec248e service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:59.627767 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b062763-08bb-4ed7-8405-0396b5ec248e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:59.636167 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6aa3fd67-a508-43b1-b74e-ec0917b965d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '563b36b0f203439ba623ca7e9cc3a212'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:59.636834 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6aa3fd67-a508-43b1-b74e-ec0917b965d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '563b36b0f203439ba623ca7e9cc3a212'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:59.638026 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b062763-08bb-4ed7-8405-0396b5ec248e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:59.639467 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b062763-08bb-4ed7-8405-0396b5ec248e service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:59.646135 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6aa3fd67-a508-43b1-b74e-ec0917b965d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '563b36b0f203439ba623ca7e9cc3a212'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:59.650995 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1335/2664] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:26:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:26:59.658431 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6aa3fd67-a508-43b1-b74e-ec0917b965d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '563b36b0f203439ba623ca7e9cc3a212'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:59.659139 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6aa3fd67-a508-43b1-b74e-ec0917b965d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '563b36b0f203439ba623ca7e9cc3a212'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:26:59.671342 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1330/2665] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:26:59 2022] DELETE /identity/v3/users/563b36b0f203439ba623ca7e9cc3a212 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:26:59.936233 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b062763-08bb-4ed7-8405-0396b5ec248e service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:59.938179 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-1b062763-08bb-4ed7-8405-0396b5ec248e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:59.953279 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b062763-08bb-4ed7-8405-0396b5ec248e service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:00.020686 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebaaec4-40e3-4d24-badd-a9092af3476b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:00.021742 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebaaec4-40e3-4d24-badd-a9092af3476b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:00.022018 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebaaec4-40e3-4d24-badd-a9092af3476b admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:00.022303 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebaaec4-40e3-4d24-badd-a9092af3476b admin admin] PATH_INFO: `/v3/projects/a482a5dd6f014193a66ffae9047b0b6a` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:00.024151 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebaaec4-40e3-4d24-badd-a9092af3476b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a482a5dd6f014193a66ffae9047b0b6a)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:00.026124 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebaaec4-40e3-4d24-badd-a9092af3476b admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:00.081257 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4ebaaec4-40e3-4d24-badd-a9092af3476b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a482a5dd6f014193a66ffae9047b0b6a', 'request_id': 'req-4ebaaec4-40e3-4d24-badd-a9092af3476b'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:00.086916 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4ebaaec4-40e3-4d24-badd-a9092af3476b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a482a5dd6f014193a66ffae9047b0b6a', 'request_id': 'req-4ebaaec4-40e3-4d24-badd-a9092af3476b'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:00.093481 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4ebaaec4-40e3-4d24-badd-a9092af3476b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a482a5dd6f014193a66ffae9047b0b6a', 'request_id': 'req-4ebaaec4-40e3-4d24-badd-a9092af3476b'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:00.108401 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4ebaaec4-40e3-4d24-badd-a9092af3476b admin admin] The token cache is being invalidate because project a482a5dd6f014193a66ffae9047b0b6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:27:00.108618 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4ebaaec4-40e3-4d24-badd-a9092af3476b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:00.109273 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4ebaaec4-40e3-4d24-badd-a9092af3476b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:00.110815 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1336/2666] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:26:59 2022] DELETE /identity/v3/projects/a482a5dd6f014193a66ffae9047b0b6a => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:27:00.115667 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa3fd67-a508-43b1-b74e-ec0917b965d9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:00.117177 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6aa3fd67-a508-43b1-b74e-ec0917b965d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:00.131045 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa3fd67-a508-43b1-b74e-ec0917b965d9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:00.194818 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb975eb4-7948-4fa4-9ee2-0e553cadf171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:00.195935 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb975eb4-7948-4fa4-9ee2-0e553cadf171 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:00.196218 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb975eb4-7948-4fa4-9ee2-0e553cadf171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:00.196496 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb975eb4-7948-4fa4-9ee2-0e553cadf171 admin admin] PATH_INFO: `/v3/users/57f9e9a9bed749cfbdd180086b4f6d56` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:00.197694 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb975eb4-7948-4fa4-9ee2-0e553cadf171 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57f9e9a9bed749cfbdd180086b4f6d56)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:00.199626 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb975eb4-7948-4fa4-9ee2-0e553cadf171 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:00.203547 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebaaec4-40e3-4d24-badd-a9092af3476b admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:00.205230 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4ebaaec4-40e3-4d24-badd-a9092af3476b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:00.206953 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:27:00.220936 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebaaec4-40e3-4d24-badd-a9092af3476b admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:00.227373 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1cf3b0-57c1-488d-a763-06b067ff2da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:00.228340 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1cf3b0-57c1-488d-a763-06b067ff2da9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:00.228693 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1cf3b0-57c1-488d-a763-06b067ff2da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:00.229204 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1cf3b0-57c1-488d-a763-06b067ff2da9 admin admin] PATH_INFO: `/v3/projects/ec1e84e6e2d542e5bef64738b90e63d2` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:00.230492 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1cf3b0-57c1-488d-a763-06b067ff2da9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec1e84e6e2d542e5bef64738b90e63d2)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:00.232509 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1cf3b0-57c1-488d-a763-06b067ff2da9 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:00.263982 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fb975eb4-7948-4fa4-9ee2-0e553cadf171 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57f9e9a9bed749cfbdd180086b4f6d56'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:00.264696 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fb975eb4-7948-4fa4-9ee2-0e553cadf171 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57f9e9a9bed749cfbdd180086b4f6d56'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:00.272940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fb975eb4-7948-4fa4-9ee2-0e553cadf171 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57f9e9a9bed749cfbdd180086b4f6d56'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:00.285389 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-7f1cf3b0-57c1-488d-a763-06b067ff2da9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec1e84e6e2d542e5bef64738b90e63d2', 'request_id': 'req-7f1cf3b0-57c1-488d-a763-06b067ff2da9'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:00.285546 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fb975eb4-7948-4fa4-9ee2-0e553cadf171 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57f9e9a9bed749cfbdd180086b4f6d56'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:00.286978 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fb975eb4-7948-4fa4-9ee2-0e553cadf171 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57f9e9a9bed749cfbdd180086b4f6d56'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:00.291672 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-7f1cf3b0-57c1-488d-a763-06b067ff2da9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec1e84e6e2d542e5bef64738b90e63d2', 'request_id': 'req-7f1cf3b0-57c1-488d-a763-06b067ff2da9'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:00.293516 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:27:00.300017 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1331/2667] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:27:00 2022] DELETE /identity/v3/users/57f9e9a9bed749cfbdd180086b4f6d56 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:27:00.300897 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-7f1cf3b0-57c1-488d-a763-06b067ff2da9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec1e84e6e2d542e5bef64738b90e63d2', 'request_id': 'req-7f1cf3b0-57c1-488d-a763-06b067ff2da9'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:00.316492 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-7f1cf3b0-57c1-488d-a763-06b067ff2da9 admin admin] The token cache is being invalidate because project ec1e84e6e2d542e5bef64738b90e63d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:27:00.316786 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-7f1cf3b0-57c1-488d-a763-06b067ff2da9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:00.317598 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-7f1cf3b0-57c1-488d-a763-06b067ff2da9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:00.319157 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1337/2668] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:27:00 2022] DELETE /identity/v3/projects/ec1e84e6e2d542e5bef64738b90e63d2 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:27:00.324215 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5d958-4d61-4473-884a-6cb3079b0d7f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:00.324504 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5d958-4d61-4473-884a-6cb3079b0d7f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:00.324781 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5d958-4d61-4473-884a-6cb3079b0d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:00.343066 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-17e5d958-4d61-4473-884a-6cb3079b0d7f None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:27:00.344537 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-17e5d958-4d61-4473-884a-6cb3079b0d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:00.429469 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1332/2669] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:27:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:27:00.431097 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e94626-2177-476f-ab08-e823f9ab5ad1 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:00.431588 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e94626-2177-476f-ab08-e823f9ab5ad1 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:00.431737 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e94626-2177-476f-ab08-e823f9ab5ad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:00.450316 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-57e94626-2177-476f-ab08-e823f9ab5ad1 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:27:00.451533 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-57e94626-2177-476f-ab08-e823f9ab5ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:00.460809 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1338/2670] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:27:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:27:00.466317 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e5d958-4d61-4473-884a-6cb3079b0d7f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:00.478131 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e5d958-4d61-4473-884a-6cb3079b0d7f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:00.483979 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358d6429-6e7d-461b-8658-1057cfa3fe97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:00.484697 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358d6429-6e7d-461b-8658-1057cfa3fe97 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:00.485074 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358d6429-6e7d-461b-8658-1057cfa3fe97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:00.485367 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358d6429-6e7d-461b-8658-1057cfa3fe97 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:00.486105 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358d6429-6e7d-461b-8658-1057cfa3fe97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:27:00.486944 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358d6429-6e7d-461b-8658-1057cfa3fe97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:00.488561 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358d6429-6e7d-461b-8658-1057cfa3fe97 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:00.504604 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1333/2671] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:27:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:27:00.506829 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e94626-2177-476f-ab08-e823f9ab5ad1 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:00.516550 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e94626-2177-476f-ab08-e823f9ab5ad1 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:00.525434 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c19e4bd-db33-4381-9b48-305fbbfb2d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:00.526106 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19e4bd-db33-4381-9b48-305fbbfb2d9a admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:00.526396 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19e4bd-db33-4381-9b48-305fbbfb2d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:00.526678 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19e4bd-db33-4381-9b48-305fbbfb2d9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:00.527378 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c19e4bd-db33-4381-9b48-305fbbfb2d9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:00.529342 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c19e4bd-db33-4381-9b48-305fbbfb2d9a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:00.556784 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1339/2672] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:27:00 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:27:00.560804 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358d6429-6e7d-461b-8658-1057cfa3fe97 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:00.571275 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358d6429-6e7d-461b-8658-1057cfa3fe97 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:00.601869 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c19e4bd-db33-4381-9b48-305fbbfb2d9a admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:00.603717 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-2c19e4bd-db33-4381-9b48-305fbbfb2d9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:00.641997 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d15ba3c-3eb2-4523-bedb-0981cbe29cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:00.642753 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d15ba3c-3eb2-4523-bedb-0981cbe29cd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:00.643201 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d15ba3c-3eb2-4523-bedb-0981cbe29cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:00.643577 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d15ba3c-3eb2-4523-bedb-0981cbe29cd6 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:00.644353 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d15ba3c-3eb2-4523-bedb-0981cbe29cd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:00.646382 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d15ba3c-3eb2-4523-bedb-0981cbe29cd6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:00.671059 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1334/2673] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:27:00 2022] POST /identity/v3/users => generated 289 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:27:00.672034 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d15ba3c-3eb2-4523-bedb-0981cbe29cd6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:00.672168 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c19e4bd-db33-4381-9b48-305fbbfb2d9a admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:00.677433 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27a2944-235d-4300-bbca-d89bdf6986b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:00.677628 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:27:00.678706 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a2944-235d-4300-bbca-d89bdf6986b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:00.678989 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a2944-235d-4300-bbca-d89bdf6986b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:00.679268 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a2944-235d-4300-bbca-d89bdf6986b2 admin admin] PATH_INFO: `/v3/projects/d275c7fc073847f69222fca8eac554fd` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:00.680463 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27a2944-235d-4300-bbca-d89bdf6986b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d275c7fc073847f69222fca8eac554fd)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:00.682401 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27a2944-235d-4300-bbca-d89bdf6986b2 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:00.685497 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d15ba3c-3eb2-4523-bedb-0981cbe29cd6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:00.690824 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea13ce2-a931-46f7-a154-47341726ad0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:00.691497 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea13ce2-a931-46f7-a154-47341726ad0b admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:00.691760 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea13ce2-a931-46f7-a154-47341726ad0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:00.692038 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea13ce2-a931-46f7-a154-47341726ad0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:00.693403 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea13ce2-a931-46f7-a154-47341726ad0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:00.694964 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea13ce2-a931-46f7-a154-47341726ad0b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:00.704861 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1335/2674] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:27:00.708670 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea13ce2-a931-46f7-a154-47341726ad0b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:00.719271 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea13ce2-a931-46f7-a154-47341726ad0b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:00.724377 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab319f-647e-4521-a6de-1ea1a07c298f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:00.725013 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab319f-647e-4521-a6de-1ea1a07c298f admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:00.725324 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab319f-647e-4521-a6de-1ea1a07c298f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:00.725605 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab319f-647e-4521-a6de-1ea1a07c298f admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:00.726321 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ab319f-647e-4521-a6de-1ea1a07c298f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:00.727876 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ab319f-647e-4521-a6de-1ea1a07c298f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:00.734579 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e27a2944-235d-4300-bbca-d89bdf6986b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd275c7fc073847f69222fca8eac554fd', 'request_id': 'req-e27a2944-235d-4300-bbca-d89bdf6986b2'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:00.738770 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1336/2675] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:27:00.742712 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e27a2944-235d-4300-bbca-d89bdf6986b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd275c7fc073847f69222fca8eac554fd', 'request_id': 'req-e27a2944-235d-4300-bbca-d89bdf6986b2'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:00.743346 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab319f-647e-4521-a6de-1ea1a07c298f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:00.747544 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e27a2944-235d-4300-bbca-d89bdf6986b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd275c7fc073847f69222fca8eac554fd', 'request_id': 'req-e27a2944-235d-4300-bbca-d89bdf6986b2'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:00.754226 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab319f-647e-4521-a6de-1ea1a07c298f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:00.761732 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e27a2944-235d-4300-bbca-d89bdf6986b2 admin admin] The token cache is being invalidate because project d275c7fc073847f69222fca8eac554fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:27:00.762019 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e27a2944-235d-4300-bbca-d89bdf6986b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:00.762658 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e27a2944-235d-4300-bbca-d89bdf6986b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:00.764423 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1340/2676] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:27:00 2022] DELETE /identity/v3/projects/d275c7fc073847f69222fca8eac554fd => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:27:00.769085 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f141f3cb-c667-4672-bc7c-5aec1d2fe6b1 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:00.769375 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f141f3cb-c667-4672-bc7c-5aec1d2fe6b1 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:00.769655 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f141f3cb-c667-4672-bc7c-5aec1d2fe6b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:00.788482 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-f141f3cb-c667-4672-bc7c-5aec1d2fe6b1 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:27:00.789727 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f141f3cb-c667-4672-bc7c-5aec1d2fe6b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:00.813653 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a3ea41-3c22-45be-84af-f6024703ac93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:00.814546 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a3ea41-3c22-45be-84af-f6024703ac93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:00.814964 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a3ea41-3c22-45be-84af-f6024703ac93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:00.815247 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a3ea41-3c22-45be-84af-f6024703ac93 admin admin] PATH_INFO: `/v3/projects/fd760a4c64954833885e2b8d400100f4/users/dc04b222c39248bebe8506dbf067efba/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:00.829490 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a3ea41-3c22-45be-84af-f6024703ac93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd760a4c64954833885e2b8d400100f4, user_id=dc04b222c39248bebe8506dbf067efba, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:00.832140 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a3ea41-3c22-45be-84af-f6024703ac93 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:00.835457 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:27:00.846775 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1337/2677] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:27:00 2022] PUT /identity/v3/projects/fd760a4c64954833885e2b8d400100f4/users/dc04b222c39248bebe8506dbf067efba/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:27:00.855662 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1341/2678] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:27:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:27:00.862324 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270495bd-7be0-47b9-bdf4-e977013b95e5 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:00.862608 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270495bd-7be0-47b9-bdf4-e977013b95e5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:00.862846 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c623b6d9-3bde-44b3-8043-aac33a252d42 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:00.862956 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270495bd-7be0-47b9-bdf4-e977013b95e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:00.863137 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c623b6d9-3bde-44b3-8043-aac33a252d42 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:00.863420 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c623b6d9-3bde-44b3-8043-aac33a252d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:00.881948 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-c623b6d9-3bde-44b3-8043-aac33a252d42 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:27:00.883274 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c623b6d9-3bde-44b3-8043-aac33a252d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:00.892693 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-270495bd-7be0-47b9-bdf4-e977013b95e5 None None] MFA Rules not processed for user `dc04b222c39248bebe8506dbf067efba`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:27:00.893910 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-270495bd-7be0-47b9-bdf4-e977013b95e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:00.946961 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1342/2679] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:27:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:27:00.950964 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c623b6d9-3bde-44b3-8043-aac33a252d42 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:00.960449 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c623b6d9-3bde-44b3-8043-aac33a252d42 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:00.962906 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1338/2680] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:27:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:27:00.968705 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309b9501-d054-4a65-afac-668f221a1254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:00.969880 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309b9501-d054-4a65-afac-668f221a1254 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:00.969880 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309b9501-d054-4a65-afac-668f221a1254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:00.969880 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309b9501-d054-4a65-afac-668f221a1254 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:00.970509 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309b9501-d054-4a65-afac-668f221a1254 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:27:00.970928 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309b9501-d054-4a65-afac-668f221a1254 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:00.972526 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309b9501-d054-4a65-afac-668f221a1254 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:00.977473 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270495bd-7be0-47b9-bdf4-e977013b95e5 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:00.978870 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-270495bd-7be0-47b9-bdf4-e977013b95e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:00.986320 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1343/2681] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:27:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:27:00.991152 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309b9501-d054-4a65-afac-668f221a1254 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:01.000768 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309b9501-d054-4a65-afac-668f221a1254 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:01.005613 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a270f8-fe8a-4441-9ed9-a897d658be16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:01.006257 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a270f8-fe8a-4441-9ed9-a897d658be16 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:01.006551 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a270f8-fe8a-4441-9ed9-a897d658be16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:01.006833 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a270f8-fe8a-4441-9ed9-a897d658be16 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:01.007521 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a270f8-fe8a-4441-9ed9-a897d658be16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:01.009464 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a270f8-fe8a-4441-9ed9-a897d658be16 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:01.039804 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1344/2682] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:27:00 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:27:01.040025 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270495bd-7be0-47b9-bdf4-e977013b95e5 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:01.041014 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a270f8-fe8a-4441-9ed9-a897d658be16 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:01.042170 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:27:01.091263 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1c4c7c-d51c-4d49-a06a-2f5c3c11b41c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '192724527d774817bc4581474ac848dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:01.091927 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c4c7c-d51c-4d49-a06a-2f5c3c11b41c service glance] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:01.092217 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c4c7c-d51c-4d49-a06a-2f5c3c11b41c service glance] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:01.092502 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c4c7c-d51c-4d49-a06a-2f5c3c11b41c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:01.105300 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a270f8-fe8a-4441-9ed9-a897d658be16 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:01.110300 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa3c3de-9df7-4031-9845-c4d33c77d851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:01.110957 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa3c3de-9df7-4031-9845-c4d33c77d851 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:01.111241 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa3c3de-9df7-4031-9845-c4d33c77d851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:01.111509 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa3c3de-9df7-4031-9845-c4d33c77d851 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:01.112229 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa3c3de-9df7-4031-9845-c4d33c77d851 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:01.114825 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa3c3de-9df7-4031-9845-c4d33c77d851 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:01.137076 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1345/2683] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:27:01 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:27:01.140622 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa3c3de-9df7-4031-9845-c4d33c77d851 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:01.151423 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa3c3de-9df7-4031-9845-c4d33c77d851 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:01.153360 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1c4c7c-d51c-4d49-a06a-2f5c3c11b41c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:01.154934 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1c4c7c-d51c-4d49-a06a-2f5c3c11b41c service glance] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:01.156712 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e151b9-0ab0-4cb2-82c4-1b948ff47b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:01.157415 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e151b9-0ab0-4cb2-82c4-1b948ff47b02 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:01.157703 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e151b9-0ab0-4cb2-82c4-1b948ff47b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:01.157990 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e151b9-0ab0-4cb2-82c4-1b948ff47b02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:01.158755 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e151b9-0ab0-4cb2-82c4-1b948ff47b02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:01.161563 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e151b9-0ab0-4cb2-82c4-1b948ff47b02 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:01.165530 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1339/2684] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:27:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:27:01.173348 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1346/2685] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:27:01.180794 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1c4c7c-d51c-4d49-a06a-2f5c3c11b41c service glance] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:01.194091 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1c4c7c-d51c-4d49-a06a-2f5c3c11b41c service glance] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:01.203376 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d04312b-3b67-47ae-83df-2b7c55740ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:01.204058 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d04312b-3b67-47ae-83df-2b7c55740ec6 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:01.204348 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d04312b-3b67-47ae-83df-2b7c55740ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:01.204628 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d04312b-3b67-47ae-83df-2b7c55740ec6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:01.205404 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d04312b-3b67-47ae-83df-2b7c55740ec6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:01.206995 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d04312b-3b67-47ae-83df-2b7c55740ec6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:01.223233 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1340/2686] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:27:01.229398 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e151b9-0ab0-4cb2-82c4-1b948ff47b02 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:01.240335 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e151b9-0ab0-4cb2-82c4-1b948ff47b02 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:01.245181 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039cdab7-3a67-4308-b1a2-ac2159c8d53c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:01.246043 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039cdab7-3a67-4308-b1a2-ac2159c8d53c admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:01.246336 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039cdab7-3a67-4308-b1a2-ac2159c8d53c admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:01.246616 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039cdab7-3a67-4308-b1a2-ac2159c8d53c admin admin] PATH_INFO: `/v3/projects/a12dfd862b50442aabea90eb685a5734/users/3eac6a73ba814d7c8fa533a3803926c6/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:01.262550 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039cdab7-3a67-4308-b1a2-ac2159c8d53c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a12dfd862b50442aabea90eb685a5734, user_id=3eac6a73ba814d7c8fa533a3803926c6, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:01.265218 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039cdab7-3a67-4308-b1a2-ac2159c8d53c admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:01.279462 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1347/2687] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:27:01 2022] PUT /identity/v3/projects/a12dfd862b50442aabea90eb685a5734/users/3eac6a73ba814d7c8fa533a3803926c6/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:27:01.291272 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda997d-2a5d-46b8-b262-bded7aee96b6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:01.291568 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda997d-2a5d-46b8-b262-bded7aee96b6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:01.291855 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda997d-2a5d-46b8-b262-bded7aee96b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:01.323259 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8dda997d-2a5d-46b8-b262-bded7aee96b6 None None] MFA Rules not processed for user `3eac6a73ba814d7c8fa533a3803926c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:27:01.324622 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8dda997d-2a5d-46b8-b262-bded7aee96b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:01.404294 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1341/2688] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:27:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:27:01.408511 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039cdab7-3a67-4308-b1a2-ac2159c8d53c admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:01.422844 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:27:01.479955 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039cdab7-3a67-4308-b1a2-ac2159c8d53c admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:01.485066 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6ee0b0-2c00-4124-95c7-998c7f6c151e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:01.485747 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ee0b0-2c00-4124-95c7-998c7f6c151e admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:01.486030 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ee0b0-2c00-4124-95c7-998c7f6c151e admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:01.486306 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ee0b0-2c00-4124-95c7-998c7f6c151e admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:01.487010 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6ee0b0-2c00-4124-95c7-998c7f6c151e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:01.488919 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6ee0b0-2c00-4124-95c7-998c7f6c151e admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:01.521080 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1348/2689] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:27:01 2022] POST /identity/v3/projects => generated 387 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:27:01.524886 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dda997d-2a5d-46b8-b262-bded7aee96b6 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:01.537781 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dda997d-2a5d-46b8-b262-bded7aee96b6 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:01.592972 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:27:01.612671 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecfcb5a-a295-4a0e-89b6-6abb9837549d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:01.613552 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecfcb5a-a295-4a0e-89b6-6abb9837549d admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:01.614318 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecfcb5a-a295-4a0e-89b6-6abb9837549d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:01.614594 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecfcb5a-a295-4a0e-89b6-6abb9837549d admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:01.615416 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecfcb5a-a295-4a0e-89b6-6abb9837549d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:01.617505 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecfcb5a-a295-4a0e-89b6-6abb9837549d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:01.640610 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1342/2690] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:27:01 2022] POST /identity/v3/users => generated 294 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:27:01.644281 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6ee0b0-2c00-4124-95c7-998c7f6c151e admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:01.655797 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6ee0b0-2c00-4124-95c7-998c7f6c151e admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:01.661500 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf735403-9994-4f5f-9e75-64e9fd98a539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:01.662325 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf735403-9994-4f5f-9e75-64e9fd98a539 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:01.662643 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf735403-9994-4f5f-9e75-64e9fd98a539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:01.663011 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf735403-9994-4f5f-9e75-64e9fd98a539 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:01.663833 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf735403-9994-4f5f-9e75-64e9fd98a539 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:01.665607 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf735403-9994-4f5f-9e75-64e9fd98a539 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:01.678073 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1349/2691] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:27:01.682404 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecfcb5a-a295-4a0e-89b6-6abb9837549d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:01.693161 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecfcb5a-a295-4a0e-89b6-6abb9837549d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:01.697784 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446b1e31-2ec8-4e85-98fb-39e6b6fed769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:01.698444 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b1e31-2ec8-4e85-98fb-39e6b6fed769 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:01.698730 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b1e31-2ec8-4e85-98fb-39e6b6fed769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:01.699017 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b1e31-2ec8-4e85-98fb-39e6b6fed769 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:01.699738 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446b1e31-2ec8-4e85-98fb-39e6b6fed769 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:01.701352 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446b1e31-2ec8-4e85-98fb-39e6b6fed769 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:01.711900 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1343/2692] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:27:01.715931 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf735403-9994-4f5f-9e75-64e9fd98a539 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:01.726545 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf735403-9994-4f5f-9e75-64e9fd98a539 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:01.731310 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5276cc4e-0516-46f7-949f-c24d8de51790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:01.732179 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5276cc4e-0516-46f7-949f-c24d8de51790 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:01.732465 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5276cc4e-0516-46f7-949f-c24d8de51790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:01.732742 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5276cc4e-0516-46f7-949f-c24d8de51790 admin admin] PATH_INFO: `/v3/projects/d3d91037abca40e8887591b2e6a24afb/users/0e7882521ddc4cbb89ca9ce2e7b30cfe/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:01.747387 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5276cc4e-0516-46f7-949f-c24d8de51790 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3d91037abca40e8887591b2e6a24afb, user_id=0e7882521ddc4cbb89ca9ce2e7b30cfe, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:01.750177 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5276cc4e-0516-46f7-949f-c24d8de51790 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:01.764356 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1350/2693] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:27:01 2022] PUT /identity/v3/projects/d3d91037abca40e8887591b2e6a24afb/users/0e7882521ddc4cbb89ca9ce2e7b30cfe/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:27:01.775217 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52957f16-1826-4b28-b6d8-445c510bcb04 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:01.775513 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52957f16-1826-4b28-b6d8-445c510bcb04 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:01.775826 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52957f16-1826-4b28-b6d8-445c510bcb04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:01.804102 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-52957f16-1826-4b28-b6d8-445c510bcb04 None None] MFA Rules not processed for user `0e7882521ddc4cbb89ca9ce2e7b30cfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:27:01.805341 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-52957f16-1826-4b28-b6d8-445c510bcb04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:01.870666 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1344/2694] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:27:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:27:01.875374 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5276cc4e-0516-46f7-949f-c24d8de51790 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:01.939222 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5276cc4e-0516-46f7-949f-c24d8de51790 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:01.944745 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5264c874-8e76-421f-804e-8258e82f3c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:01.945435 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5264c874-8e76-421f-804e-8258e82f3c99 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:01.945724 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5264c874-8e76-421f-804e-8258e82f3c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:01.945999 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5264c874-8e76-421f-804e-8258e82f3c99 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:01.946705 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5264c874-8e76-421f-804e-8258e82f3c99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:01.948604 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5264c874-8e76-421f-804e-8258e82f3c99 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:01.976109 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1351/2695] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:27:01 2022] POST /identity/v3/projects => generated 387 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:27:01.980698 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52957f16-1826-4b28-b6d8-445c510bcb04 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:01.995164 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52957f16-1826-4b28-b6d8-445c510bcb04 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:02.061630 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2be04b-a576-44a3-98cb-09a06fc79f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:02.062312 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2be04b-a576-44a3-98cb-09a06fc79f15 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:02.062601 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2be04b-a576-44a3-98cb-09a06fc79f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:02.062872 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2be04b-a576-44a3-98cb-09a06fc79f15 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:02.063560 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2be04b-a576-44a3-98cb-09a06fc79f15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:02.065514 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2be04b-a576-44a3-98cb-09a06fc79f15 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:02.086138 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1345/2696] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:27:01 2022] POST /identity/v3/users => generated 300 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:27:02.090165 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5264c874-8e76-421f-804e-8258e82f3c99 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:02.101483 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5264c874-8e76-421f-804e-8258e82f3c99 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:02.106277 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d81bb-c36d-43f1-a6ed-8a5bdbbe8272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:02.106949 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d81bb-c36d-43f1-a6ed-8a5bdbbe8272 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:02.107237 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d81bb-c36d-43f1-a6ed-8a5bdbbe8272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:02.107545 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d81bb-c36d-43f1-a6ed-8a5bdbbe8272 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:02.108309 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618d81bb-c36d-43f1-a6ed-8a5bdbbe8272 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:02.109962 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618d81bb-c36d-43f1-a6ed-8a5bdbbe8272 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:02.121129 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1352/2697] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:27:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:27:02.125326 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2be04b-a576-44a3-98cb-09a06fc79f15 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:02.136216 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2be04b-a576-44a3-98cb-09a06fc79f15 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:02.140899 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ed8ff7-7686-43d9-980b-754099d056b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:02.141829 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed8ff7-7686-43d9-980b-754099d056b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:02.142120 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed8ff7-7686-43d9-980b-754099d056b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:02.142400 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed8ff7-7686-43d9-980b-754099d056b3 admin admin] PATH_INFO: `/v3/projects/a59cb3758f6b4f52a14b98067224bb4c/users/e3dfda1d7fc84700a2c3031cc44950cc/roles/f36e053b442540f08041ac7878760f65` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:02.158722 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ed8ff7-7686-43d9-980b-754099d056b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a59cb3758f6b4f52a14b98067224bb4c, user_id=e3dfda1d7fc84700a2c3031cc44950cc, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:02.162304 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ed8ff7-7686-43d9-980b-754099d056b3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:02.176313 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1346/2698] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:27:02 2022] PUT /identity/v3/projects/a59cb3758f6b4f52a14b98067224bb4c/users/e3dfda1d7fc84700a2c3031cc44950cc/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:27:02.180202 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d81bb-c36d-43f1-a6ed-8a5bdbbe8272 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:02.191095 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d81bb-c36d-43f1-a6ed-8a5bdbbe8272 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:02.230845 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:27:02.261092 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de331d63-9b64-4146-980c-3e4f317cdf11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:02.261879 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de331d63-9b64-4146-980c-3e4f317cdf11 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:02.262168 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de331d63-9b64-4146-980c-3e4f317cdf11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:02.262450 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de331d63-9b64-4146-980c-3e4f317cdf11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:02.263184 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de331d63-9b64-4146-980c-3e4f317cdf11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:02.264797 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de331d63-9b64-4146-980c-3e4f317cdf11 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:02.277077 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1353/2699] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:27:02 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:27:02.280704 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ed8ff7-7686-43d9-980b-754099d056b3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:02.295221 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ed8ff7-7686-43d9-980b-754099d056b3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:02.299836 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3417e9a-a66a-4a70-9ce4-065b83c56827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:02.300895 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3417e9a-a66a-4a70-9ce4-065b83c56827 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:02.301349 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3417e9a-a66a-4a70-9ce4-065b83c56827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:02.301778 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3417e9a-a66a-4a70-9ce4-065b83c56827 admin admin] PATH_INFO: `/v3/projects/a59cb3758f6b4f52a14b98067224bb4c/users/e3dfda1d7fc84700a2c3031cc44950cc/roles/f36e053b442540f08041ac7878760f65` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:02.304089 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3417e9a-a66a-4a70-9ce4-065b83c56827 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a59cb3758f6b4f52a14b98067224bb4c, user_id=e3dfda1d7fc84700a2c3031cc44950cc, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:02.308282 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3417e9a-a66a-4a70-9ce4-065b83c56827 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:02.318844 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1347/2700] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:27:02 2022] PUT /identity/v3/projects/a59cb3758f6b4f52a14b98067224bb4c/users/e3dfda1d7fc84700a2c3031cc44950cc/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:27:02.330334 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52610a0f-9bfd-46d7-aee2-a5f85a48aa4a None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:02.330680 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52610a0f-9bfd-46d7-aee2-a5f85a48aa4a None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:02.331010 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52610a0f-9bfd-46d7-aee2-a5f85a48aa4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:02.368043 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-52610a0f-9bfd-46d7-aee2-a5f85a48aa4a None None] MFA Rules not processed for user `e3dfda1d7fc84700a2c3031cc44950cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:27:02.369458 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-52610a0f-9bfd-46d7-aee2-a5f85a48aa4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:02.448083 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1354/2701] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:27:02 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:27:02.459797 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3417e9a-a66a-4a70-9ce4-065b83c56827 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:02.461700 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c3417e9a-a66a-4a70-9ce4-065b83c56827 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:02.467147 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:27:02.525965 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3417e9a-a66a-4a70-9ce4-065b83c56827 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:02.529609 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d21eb52-f0c2-4cbe-8567-7cf40854b721 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:02.530269 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d21eb52-f0c2-4cbe-8567-7cf40854b721 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:02.530556 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d21eb52-f0c2-4cbe-8567-7cf40854b721 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:02.531108 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d21eb52-f0c2-4cbe-8567-7cf40854b721 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:02.541259 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d21eb52-f0c2-4cbe-8567-7cf40854b721 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:02.542779 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d21eb52-f0c2-4cbe-8567-7cf40854b721 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:02.553328 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1348/2702] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:27:02 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:27:03.092652 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52610a0f-9bfd-46d7-aee2-a5f85a48aa4a None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:03.094347 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-52610a0f-9bfd-46d7-aee2-a5f85a48aa4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:03.106445 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52610a0f-9bfd-46d7-aee2-a5f85a48aa4a None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:03.110030 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0e7541-9c39-4aaa-97cb-e2bd14a20a54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:03.110671 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0e7541-9c39-4aaa-97cb-e2bd14a20a54 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:03.110961 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0e7541-9c39-4aaa-97cb-e2bd14a20a54 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:03.111244 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0e7541-9c39-4aaa-97cb-e2bd14a20a54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:03.168335 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0e7541-9c39-4aaa-97cb-e2bd14a20a54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:03.169819 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0e7541-9c39-4aaa-97cb-e2bd14a20a54 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:03.178721 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1355/2703] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:27:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:27:12.548746 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=25 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:27:13.174633 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=75 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:27:13.857341 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d21eb52-f0c2-4cbe-8567-7cf40854b721 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:13.858777 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3d21eb52-f0c2-4cbe-8567-7cf40854b721 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:13.929327 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d21eb52-f0c2-4cbe-8567-7cf40854b721 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:13.934546 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d91650-7453-4bde-9277-fb654b71cf59 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '293c695f15c8437d81a619f840318f44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:13.935197 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d91650-7453-4bde-9277-fb654b71cf59 service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:13.935480 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d91650-7453-4bde-9277-fb654b71cf59 service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:13.935780 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d91650-7453-4bde-9277-fb654b71cf59 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:13.937621 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c0d91650-7453-4bde-9277-fb654b71cf59 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:13.949660 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d91650-7453-4bde-9277-fb654b71cf59 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:13.951217 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d91650-7453-4bde-9277-fb654b71cf59 service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:13.962412 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1349/2704] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:27:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:27:14.595538 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0e7541-9c39-4aaa-97cb-e2bd14a20a54 service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:14.604644 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0e7541-9c39-4aaa-97cb-e2bd14a20a54 service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:14.608424 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594e1eb2-e7df-4a70-b7db-f71921dd44cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:14.609100 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e1eb2-e7df-4a70-b7db-f71921dd44cd service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:14.609393 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e1eb2-e7df-4a70-b7db-f71921dd44cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:14.609674 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e1eb2-e7df-4a70-b7db-f71921dd44cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:14.669133 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594e1eb2-e7df-4a70-b7db-f71921dd44cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:14.670587 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594e1eb2-e7df-4a70-b7db-f71921dd44cd service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:14.681376 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1356/2705] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:27:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:27:16.783947 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d91650-7453-4bde-9277-fb654b71cf59 service placement] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:16.848165 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d91650-7453-4bde-9277-fb654b71cf59 service placement] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:16.853442 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190b48b9-1f48-4a17-bb3d-d72005e7e283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:16.854315 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190b48b9-1f48-4a17-bb3d-d72005e7e283 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:16.854604 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190b48b9-1f48-4a17-bb3d-d72005e7e283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:16.854886 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190b48b9-1f48-4a17-bb3d-d72005e7e283 admin admin] PATH_INFO: `/v3/users/3eac6a73ba814d7c8fa533a3803926c6` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:16.856087 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190b48b9-1f48-4a17-bb3d-d72005e7e283 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3eac6a73ba814d7c8fa533a3803926c6)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:16.858049 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190b48b9-1f48-4a17-bb3d-d72005e7e283 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:16.909386 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-190b48b9-1f48-4a17-bb3d-d72005e7e283 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3eac6a73ba814d7c8fa533a3803926c6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:16.910053 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-190b48b9-1f48-4a17-bb3d-d72005e7e283 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3eac6a73ba814d7c8fa533a3803926c6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:16.921770 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-190b48b9-1f48-4a17-bb3d-d72005e7e283 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3eac6a73ba814d7c8fa533a3803926c6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:16.935302 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-190b48b9-1f48-4a17-bb3d-d72005e7e283 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3eac6a73ba814d7c8fa533a3803926c6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:16.936198 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-190b48b9-1f48-4a17-bb3d-d72005e7e283 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3eac6a73ba814d7c8fa533a3803926c6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:16.956032 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1350/2706] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:27:16 2022] DELETE /identity/v3/users/3eac6a73ba814d7c8fa533a3803926c6 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:27:16.961848 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0cddfe-9873-43d8-a621-434801aeda1f None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:16.962354 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0cddfe-9873-43d8-a621-434801aeda1f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:16.962464 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0cddfe-9873-43d8-a621-434801aeda1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:16.981020 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-8a0cddfe-9873-43d8-a621-434801aeda1f None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:27:16.982371 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-8a0cddfe-9873-43d8-a621-434801aeda1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:17.043513 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1357/2707] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:27:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:27:17.053357 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190b48b9-1f48-4a17-bb3d-d72005e7e283 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:17.054798 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-190b48b9-1f48-4a17-bb3d-d72005e7e283 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:17.065982 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:27:17.119377 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190b48b9-1f48-4a17-bb3d-d72005e7e283 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:17.123376 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcd2c0e-73de-4b47-9566-c8d12fec3329 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:17.124033 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd2c0e-73de-4b47-9566-c8d12fec3329 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:17.124323 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd2c0e-73de-4b47-9566-c8d12fec3329 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:17.124608 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd2c0e-73de-4b47-9566-c8d12fec3329 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:17.138466 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcd2c0e-73de-4b47-9566-c8d12fec3329 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:17.139996 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcd2c0e-73de-4b47-9566-c8d12fec3329 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:17.150835 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1351/2708] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:27:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:27:17.452450 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0cddfe-9873-43d8-a621-434801aeda1f None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:17.453998 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-8a0cddfe-9873-43d8-a621-434801aeda1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:17.468566 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0cddfe-9873-43d8-a621-434801aeda1f None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:17.474109 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac97128-9ce1-4949-a78b-c8c4eaf6c303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:17.475523 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac97128-9ce1-4949-a78b-c8c4eaf6c303 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:17.475809 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac97128-9ce1-4949-a78b-c8c4eaf6c303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:17.476088 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac97128-9ce1-4949-a78b-c8c4eaf6c303 admin admin] PATH_INFO: `/v3/projects/a12dfd862b50442aabea90eb685a5734` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:17.477342 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac97128-9ce1-4949-a78b-c8c4eaf6c303 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a12dfd862b50442aabea90eb685a5734)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:17.479267 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac97128-9ce1-4949-a78b-c8c4eaf6c303 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:17.536350 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-5ac97128-9ce1-4949-a78b-c8c4eaf6c303 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a12dfd862b50442aabea90eb685a5734', 'request_id': 'req-5ac97128-9ce1-4949-a78b-c8c4eaf6c303'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:17.546781 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-5ac97128-9ce1-4949-a78b-c8c4eaf6c303 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a12dfd862b50442aabea90eb685a5734', 'request_id': 'req-5ac97128-9ce1-4949-a78b-c8c4eaf6c303'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:17.553770 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-5ac97128-9ce1-4949-a78b-c8c4eaf6c303 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a12dfd862b50442aabea90eb685a5734', 'request_id': 'req-5ac97128-9ce1-4949-a78b-c8c4eaf6c303'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:17.566986 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-5ac97128-9ce1-4949-a78b-c8c4eaf6c303 admin admin] The token cache is being invalidate because project a12dfd862b50442aabea90eb685a5734 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:27:17.567255 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-5ac97128-9ce1-4949-a78b-c8c4eaf6c303 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:17.567981 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-5ac97128-9ce1-4949-a78b-c8c4eaf6c303 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:17.570572 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1358/2709] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:27:17 2022] DELETE /identity/v3/projects/a12dfd862b50442aabea90eb685a5734 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:27:17.574493 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcd2c0e-73de-4b47-9566-c8d12fec3329 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:17.584439 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcd2c0e-73de-4b47-9566-c8d12fec3329 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:17.649710 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f1fe64-2fa6-4d8b-8f99-1f4d68bcfef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:17.650598 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f1fe64-2fa6-4d8b-8f99-1f4d68bcfef1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:17.650886 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f1fe64-2fa6-4d8b-8f99-1f4d68bcfef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:17.651168 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f1fe64-2fa6-4d8b-8f99-1f4d68bcfef1 admin admin] PATH_INFO: `/v3/users/0e7882521ddc4cbb89ca9ce2e7b30cfe` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:17.652368 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f1fe64-2fa6-4d8b-8f99-1f4d68bcfef1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e7882521ddc4cbb89ca9ce2e7b30cfe)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:17.654369 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f1fe64-2fa6-4d8b-8f99-1f4d68bcfef1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:17.709173 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e7f1fe64-2fa6-4d8b-8f99-1f4d68bcfef1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e7882521ddc4cbb89ca9ce2e7b30cfe'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:17.711162 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e7f1fe64-2fa6-4d8b-8f99-1f4d68bcfef1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e7882521ddc4cbb89ca9ce2e7b30cfe'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:17.721115 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e7f1fe64-2fa6-4d8b-8f99-1f4d68bcfef1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e7882521ddc4cbb89ca9ce2e7b30cfe'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:17.741845 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e7f1fe64-2fa6-4d8b-8f99-1f4d68bcfef1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e7882521ddc4cbb89ca9ce2e7b30cfe'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:17.742680 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e7f1fe64-2fa6-4d8b-8f99-1f4d68bcfef1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e7882521ddc4cbb89ca9ce2e7b30cfe'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:17.759934 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1352/2710] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:27:17 2022] DELETE /identity/v3/users/0e7882521ddc4cbb89ca9ce2e7b30cfe => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:27:18.023897 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac97128-9ce1-4949-a78b-c8c4eaf6c303 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:18.025398 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-5ac97128-9ce1-4949-a78b-c8c4eaf6c303 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:18.040929 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac97128-9ce1-4949-a78b-c8c4eaf6c303 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:18.052699 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f1fe64-2fa6-4d8b-8f99-1f4d68bcfef1 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:18.055244 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e7f1fe64-2fa6-4d8b-8f99-1f4d68bcfef1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:18.068369 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:27:18.103498 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:27:18.116980 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be41b8e-5b42-49a8-83ed-eac93f3eb333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:18.117896 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be41b8e-5b42-49a8-83ed-eac93f3eb333 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:18.118177 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be41b8e-5b42-49a8-83ed-eac93f3eb333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:18.118638 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be41b8e-5b42-49a8-83ed-eac93f3eb333 admin admin] PATH_INFO: `/v3/users/e8f16f9e9c7446b3b622d206b4545ea8` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:18.119926 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be41b8e-5b42-49a8-83ed-eac93f3eb333 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8f16f9e9c7446b3b622d206b4545ea8)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:18.121889 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be41b8e-5b42-49a8-83ed-eac93f3eb333 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:18.139006 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f1fe64-2fa6-4d8b-8f99-1f4d68bcfef1 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:18.143794 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113f635c-f5a9-4d30-8770-2696eed074bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:18.144697 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f635c-f5a9-4d30-8770-2696eed074bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:18.144984 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f635c-f5a9-4d30-8770-2696eed074bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:18.145298 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f635c-f5a9-4d30-8770-2696eed074bf admin admin] PATH_INFO: `/v3/projects/d3d91037abca40e8887591b2e6a24afb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:18.146488 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113f635c-f5a9-4d30-8770-2696eed074bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3d91037abca40e8887591b2e6a24afb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:18.148669 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113f635c-f5a9-4d30-8770-2696eed074bf admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:18.174222 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4be41b8e-5b42-49a8-83ed-eac93f3eb333 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8f16f9e9c7446b3b622d206b4545ea8'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:18.184962 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4be41b8e-5b42-49a8-83ed-eac93f3eb333 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8f16f9e9c7446b3b622d206b4545ea8'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:18.185755 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4be41b8e-5b42-49a8-83ed-eac93f3eb333 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8f16f9e9c7446b3b622d206b4545ea8'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:18.195147 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4be41b8e-5b42-49a8-83ed-eac93f3eb333 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8f16f9e9c7446b3b622d206b4545ea8'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:18.205439 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-113f635c-f5a9-4d30-8770-2696eed074bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3d91037abca40e8887591b2e6a24afb', 'request_id': 'req-113f635c-f5a9-4d30-8770-2696eed074bf'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:18.208174 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4be41b8e-5b42-49a8-83ed-eac93f3eb333 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8f16f9e9c7446b3b622d206b4545ea8'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:18.215162 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-113f635c-f5a9-4d30-8770-2696eed074bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3d91037abca40e8887591b2e6a24afb', 'request_id': 'req-113f635c-f5a9-4d30-8770-2696eed074bf'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:18.216446 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1359/2711] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:27:18 2022] DELETE /identity/v3/users/e8f16f9e9c7446b3b622d206b4545ea8 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:27:18.222187 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaca4c3-e612-4df4-b496-a3f57d6a354c None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:18.222425 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaca4c3-e612-4df4-b496-a3f57d6a354c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:18.222711 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaca4c3-e612-4df4-b496-a3f57d6a354c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:18.223735 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-113f635c-f5a9-4d30-8770-2696eed074bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3d91037abca40e8887591b2e6a24afb', 'request_id': 'req-113f635c-f5a9-4d30-8770-2696eed074bf'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:18.238011 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-113f635c-f5a9-4d30-8770-2696eed074bf admin admin] The token cache is being invalidate because project d3d91037abca40e8887591b2e6a24afb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:27:18.238563 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-113f635c-f5a9-4d30-8770-2696eed074bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:18.239242 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-113f635c-f5a9-4d30-8770-2696eed074bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:18.241512 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1353/2712] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:27:18 2022] DELETE /identity/v3/projects/d3d91037abca40e8887591b2e6a24afb => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:27:18.243057 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-ffaca4c3-e612-4df4-b496-a3f57d6a354c None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:27:18.244368 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ffaca4c3-e612-4df4-b496-a3f57d6a354c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:18.246623 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113f635c-f5a9-4d30-8770-2696eed074bf admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:18.247956 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-113f635c-f5a9-4d30-8770-2696eed074bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:18.315919 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1360/2713] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:27:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:27:18.327029 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaca4c3-e612-4df4-b496-a3f57d6a354c None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:18.328717 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ffaca4c3-e612-4df4-b496-a3f57d6a354c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:18.334027 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113f635c-f5a9-4d30-8770-2696eed074bf admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:18.340082 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af386340-2708-45d0-9f47-380b97f243fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:18.340920 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af386340-2708-45d0-9f47-380b97f243fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:18.341217 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af386340-2708-45d0-9f47-380b97f243fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:18.341524 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af386340-2708-45d0-9f47-380b97f243fa admin admin] PATH_INFO: `/v3/users/e3dfda1d7fc84700a2c3031cc44950cc` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:18.342735 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af386340-2708-45d0-9f47-380b97f243fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3dfda1d7fc84700a2c3031cc44950cc)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:18.344925 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af386340-2708-45d0-9f47-380b97f243fa admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:18.389312 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:27:18.412319 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaca4c3-e612-4df4-b496-a3f57d6a354c None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:18.415585 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-af386340-2708-45d0-9f47-380b97f243fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3dfda1d7fc84700a2c3031cc44950cc'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:18.416570 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-af386340-2708-45d0-9f47-380b97f243fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3dfda1d7fc84700a2c3031cc44950cc'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:18.418393 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a5e71-b466-49d5-b1d4-a9b36538cb67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:18.419050 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a5e71-b466-49d5-b1d4-a9b36538cb67 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:18.419408 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a5e71-b466-49d5-b1d4-a9b36538cb67 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:18.419664 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a5e71-b466-49d5-b1d4-a9b36538cb67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:18.426855 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-af386340-2708-45d0-9f47-380b97f243fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3dfda1d7fc84700a2c3031cc44950cc'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:18.432800 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851a5e71-b466-49d5-b1d4-a9b36538cb67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:18.434339 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851a5e71-b466-49d5-b1d4-a9b36538cb67 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:18.442342 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:27:18.448583 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1361/2714] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:27:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:27:18.457259 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-af386340-2708-45d0-9f47-380b97f243fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3dfda1d7fc84700a2c3031cc44950cc'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:18.458386 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-af386340-2708-45d0-9f47-380b97f243fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3dfda1d7fc84700a2c3031cc44950cc'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:18.474649 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1354/2715] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:27:18 2022] DELETE /identity/v3/users/e3dfda1d7fc84700a2c3031cc44950cc => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:27:18.858902 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a5e71-b466-49d5-b1d4-a9b36538cb67 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:18.860898 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-851a5e71-b466-49d5-b1d4-a9b36538cb67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:18.865341 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af386340-2708-45d0-9f47-380b97f243fa admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:18.867050 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-af386340-2708-45d0-9f47-380b97f243fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:18.877292 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a5e71-b466-49d5-b1d4-a9b36538cb67 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:18.952119 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c08435-2c5f-404f-8ea8-a991b95ef59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:18.952997 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c08435-2c5f-404f-8ea8-a991b95ef59a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:18.953406 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c08435-2c5f-404f-8ea8-a991b95ef59a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:18.953681 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c08435-2c5f-404f-8ea8-a991b95ef59a admin admin] PATH_INFO: `/v3/projects/d97f48dbaf9f4f0cbff1f22093b5746c` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:18.955180 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c08435-2c5f-404f-8ea8-a991b95ef59a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d97f48dbaf9f4f0cbff1f22093b5746c)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:18.957210 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c08435-2c5f-404f-8ea8-a991b95ef59a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:18.964443 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af386340-2708-45d0-9f47-380b97f243fa admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:18.969742 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6ce5d6-fc72-4ae8-a672-c3f9ef316f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:18.970594 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6ce5d6-fc72-4ae8-a672-c3f9ef316f86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:18.970875 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6ce5d6-fc72-4ae8-a672-c3f9ef316f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:18.971146 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6ce5d6-fc72-4ae8-a672-c3f9ef316f86 admin admin] PATH_INFO: `/v3/projects/a59cb3758f6b4f52a14b98067224bb4c` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:18.972341 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6ce5d6-fc72-4ae8-a672-c3f9ef316f86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a59cb3758f6b4f52a14b98067224bb4c)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:18.974265 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6ce5d6-fc72-4ae8-a672-c3f9ef316f86 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:19.016204 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b2c08435-2c5f-404f-8ea8-a991b95ef59a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd97f48dbaf9f4f0cbff1f22093b5746c', 'request_id': 'req-b2c08435-2c5f-404f-8ea8-a991b95ef59a'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:19.021774 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b2c08435-2c5f-404f-8ea8-a991b95ef59a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd97f48dbaf9f4f0cbff1f22093b5746c', 'request_id': 'req-b2c08435-2c5f-404f-8ea8-a991b95ef59a'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:19.026113 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2a6ce5d6-fc72-4ae8-a672-c3f9ef316f86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a59cb3758f6b4f52a14b98067224bb4c', 'request_id': 'req-2a6ce5d6-fc72-4ae8-a672-c3f9ef316f86'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:19.029170 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b2c08435-2c5f-404f-8ea8-a991b95ef59a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd97f48dbaf9f4f0cbff1f22093b5746c', 'request_id': 'req-b2c08435-2c5f-404f-8ea8-a991b95ef59a'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:19.031775 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2a6ce5d6-fc72-4ae8-a672-c3f9ef316f86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a59cb3758f6b4f52a14b98067224bb4c', 'request_id': 'req-2a6ce5d6-fc72-4ae8-a672-c3f9ef316f86'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:19.039971 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2a6ce5d6-fc72-4ae8-a672-c3f9ef316f86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a59cb3758f6b4f52a14b98067224bb4c', 'request_id': 'req-2a6ce5d6-fc72-4ae8-a672-c3f9ef316f86'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:19.046823 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b2c08435-2c5f-404f-8ea8-a991b95ef59a admin admin] The token cache is being invalidate because project d97f48dbaf9f4f0cbff1f22093b5746c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:27:19.047116 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b2c08435-2c5f-404f-8ea8-a991b95ef59a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:19.047891 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b2c08435-2c5f-404f-8ea8-a991b95ef59a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:19.049764 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1362/2716] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:27:18 2022] DELETE /identity/v3/projects/d97f48dbaf9f4f0cbff1f22093b5746c => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:27:19.056412 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dddbb2-6fd2-4908-b5b2-f548e9ca34aa None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:19.056780 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dddbb2-6fd2-4908-b5b2-f548e9ca34aa None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:19.057086 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dddbb2-6fd2-4908-b5b2-f548e9ca34aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:19.058151 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2a6ce5d6-fc72-4ae8-a672-c3f9ef316f86 admin admin] The token cache is being invalidate because project a59cb3758f6b4f52a14b98067224bb4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:27:19.058911 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2a6ce5d6-fc72-4ae8-a672-c3f9ef316f86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:19.059483 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2a6ce5d6-fc72-4ae8-a672-c3f9ef316f86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:19.061382 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1355/2717] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:27:18 2022] DELETE /identity/v3/projects/a59cb3758f6b4f52a14b98067224bb4c => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:27:19.069975 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f93f614-8722-4cf2-b83b-e089bf30671d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:19.070438 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f93f614-8722-4cf2-b83b-e089bf30671d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:19.070899 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f93f614-8722-4cf2-b83b-e089bf30671d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:19.083778 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-c5dddbb2-6fd2-4908-b5b2-f548e9ca34aa None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:27:19.086382 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c5dddbb2-6fd2-4908-b5b2-f548e9ca34aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:19.098971 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9f93f614-8722-4cf2-b83b-e089bf30671d None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:27:19.101498 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9f93f614-8722-4cf2-b83b-e089bf30671d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:19.166473 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1363/2718] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:27:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:27:19.172774 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1356/2719] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:27:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:27:19.180870 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d480bd9-6e05-4d31-af49-7e16ce50386c None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:19.181187 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d480bd9-6e05-4d31-af49-7e16ce50386c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:19.181475 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d480bd9-6e05-4d31-af49-7e16ce50386c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:19.181728 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c4b37-4a91-44c5-8d55-5daaf138395a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:19.182022 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c4b37-4a91-44c5-8d55-5daaf138395a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:19.182303 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c4b37-4a91-44c5-8d55-5daaf138395a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:19.200908 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-0d480bd9-6e05-4d31-af49-7e16ce50386c None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:27:19.201220 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-391c4b37-4a91-44c5-8d55-5daaf138395a None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:27:19.202155 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0d480bd9-6e05-4d31-af49-7e16ce50386c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:19.202450 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-391c4b37-4a91-44c5-8d55-5daaf138395a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:19.210836 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1364/2720] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:27:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:27:19.211341 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1357/2721] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:27:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:27:19.216097 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d480bd9-6e05-4d31-af49-7e16ce50386c None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:19.216432 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391c4b37-4a91-44c5-8d55-5daaf138395a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:19.223783 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:27:19.226699 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d480bd9-6e05-4d31-af49-7e16ce50386c None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:19.227879 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391c4b37-4a91-44c5-8d55-5daaf138395a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:19.232642 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ad82ba-ea0a-4be8-b4d0-2c3f147ca49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:19.233499 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad82ba-ea0a-4be8-b4d0-2c3f147ca49a admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:19.233826 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad82ba-ea0a-4be8-b4d0-2c3f147ca49a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:19.234116 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad82ba-ea0a-4be8-b4d0-2c3f147ca49a admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:19.235029 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ad82ba-ea0a-4be8-b4d0-2c3f147ca49a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:27:19.235331 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5860305d-7e71-4116-a623-45c5e1f8a00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:19.235549 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ad82ba-ea0a-4be8-b4d0-2c3f147ca49a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:19.236058 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5860305d-7e71-4116-a623-45c5e1f8a00d admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:19.236541 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5860305d-7e71-4116-a623-45c5e1f8a00d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:19.236834 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5860305d-7e71-4116-a623-45c5e1f8a00d admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:19.237318 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ad82ba-ea0a-4be8-b4d0-2c3f147ca49a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:19.237760 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5860305d-7e71-4116-a623-45c5e1f8a00d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:27:19.238139 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5860305d-7e71-4116-a623-45c5e1f8a00d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:19.239914 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5860305d-7e71-4116-a623-45c5e1f8a00d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:19.240094 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:27:19.252852 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1358/2722] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:27:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:27:19.253238 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1365/2723] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:27:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:27:19.258004 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5860305d-7e71-4116-a623-45c5e1f8a00d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:19.258165 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ad82ba-ea0a-4be8-b4d0-2c3f147ca49a admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:19.268216 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5860305d-7e71-4116-a623-45c5e1f8a00d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:19.268809 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ad82ba-ea0a-4be8-b4d0-2c3f147ca49a admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:19.273146 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc3c5c3-55f1-4b88-8c77-010f7a3325e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:19.273588 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8dd6a2-aedb-4e1c-b77e-89c515833cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:19.273802 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc3c5c3-55f1-4b88-8c77-010f7a3325e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:19.274086 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc3c5c3-55f1-4b88-8c77-010f7a3325e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:19.274242 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8dd6a2-aedb-4e1c-b77e-89c515833cd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:19.274344 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc3c5c3-55f1-4b88-8c77-010f7a3325e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:19.274534 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8dd6a2-aedb-4e1c-b77e-89c515833cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:19.274818 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8dd6a2-aedb-4e1c-b77e-89c515833cd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:19.275038 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc3c5c3-55f1-4b88-8c77-010f7a3325e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:19.275513 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8dd6a2-aedb-4e1c-b77e-89c515833cd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:19.277057 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc3c5c3-55f1-4b88-8c77-010f7a3325e3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:19.277436 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8dd6a2-aedb-4e1c-b77e-89c515833cd9 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:19.310276 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1366/2724] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:27:19 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:27:19.310603 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1359/2725] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:27:19 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:27:19.314361 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8dd6a2-aedb-4e1c-b77e-89c515833cd9 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:19.314594 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc3c5c3-55f1-4b88-8c77-010f7a3325e3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:19.380353 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc3c5c3-55f1-4b88-8c77-010f7a3325e3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:19.385774 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa92c4dc-dbcf-443c-b95c-05dd28bdf79e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:19.386452 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa92c4dc-dbcf-443c-b95c-05dd28bdf79e admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:19.386737 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa92c4dc-dbcf-443c-b95c-05dd28bdf79e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:19.387019 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa92c4dc-dbcf-443c-b95c-05dd28bdf79e admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:19.387714 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa92c4dc-dbcf-443c-b95c-05dd28bdf79e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:19.389771 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa92c4dc-dbcf-443c-b95c-05dd28bdf79e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:19.392461 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8dd6a2-aedb-4e1c-b77e-89c515833cd9 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:19.397948 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf2706-6e3b-4b32-92e0-aa0c688402fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:19.398632 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf2706-6e3b-4b32-92e0-aa0c688402fe admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:19.399064 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf2706-6e3b-4b32-92e0-aa0c688402fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:19.399367 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf2706-6e3b-4b32-92e0-aa0c688402fe admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:19.400107 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cf2706-6e3b-4b32-92e0-aa0c688402fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:19.401988 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cf2706-6e3b-4b32-92e0-aa0c688402fe admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:19.411109 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1360/2726] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:27:19 2022] POST /identity/v3/users => generated 286 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:27:19.415484 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa92c4dc-dbcf-443c-b95c-05dd28bdf79e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:19.423228 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1367/2727] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:27:19 2022] POST /identity/v3/users => generated 287 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:27:19.427369 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf2706-6e3b-4b32-92e0-aa0c688402fe admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:19.429379 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa92c4dc-dbcf-443c-b95c-05dd28bdf79e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:19.433769 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51e4daf-5282-4385-8e9f-a15373955734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:19.434449 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e4daf-5282-4385-8e9f-a15373955734 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:19.434739 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e4daf-5282-4385-8e9f-a15373955734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:19.435028 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e4daf-5282-4385-8e9f-a15373955734 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:19.435765 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51e4daf-5282-4385-8e9f-a15373955734 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:19.437489 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51e4daf-5282-4385-8e9f-a15373955734 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:19.438519 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf2706-6e3b-4b32-92e0-aa0c688402fe admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:19.445607 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23dda58-cc55-4f8e-849f-c15f391d1782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:19.446407 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23dda58-cc55-4f8e-849f-c15f391d1782 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:19.446973 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23dda58-cc55-4f8e-849f-c15f391d1782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:19.446973 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23dda58-cc55-4f8e-849f-c15f391d1782 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:19.447687 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23dda58-cc55-4f8e-849f-c15f391d1782 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:19.449220 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1361/2728] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:27:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:27:19.449357 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23dda58-cc55-4f8e-849f-c15f391d1782 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:19.453530 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51e4daf-5282-4385-8e9f-a15373955734 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:19.462647 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1368/2729] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:27:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:27:19.466231 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23dda58-cc55-4f8e-849f-c15f391d1782 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:19.471435 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51e4daf-5282-4385-8e9f-a15373955734 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:19.477523 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e39224f-0766-4c44-930a-c903cf6fc904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:19.478229 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e39224f-0766-4c44-930a-c903cf6fc904 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:19.478543 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e39224f-0766-4c44-930a-c903cf6fc904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:19.478696 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23dda58-cc55-4f8e-849f-c15f391d1782 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:19.478854 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e39224f-0766-4c44-930a-c903cf6fc904 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:19.479544 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e39224f-0766-4c44-930a-c903cf6fc904 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:19.481214 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e39224f-0766-4c44-930a-c903cf6fc904 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:19.484525 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ebffbc-fe51-46ae-bb40-f4182ce6dccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:19.485338 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ebffbc-fe51-46ae-bb40-f4182ce6dccf admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:19.485628 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ebffbc-fe51-46ae-bb40-f4182ce6dccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:19.485909 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ebffbc-fe51-46ae-bb40-f4182ce6dccf admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:19.487934 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ebffbc-fe51-46ae-bb40-f4182ce6dccf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:19.488283 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ebffbc-fe51-46ae-bb40-f4182ce6dccf admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:19.495593 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1362/2730] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:27:19 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:27:19.496384 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e39224f-0766-4c44-930a-c903cf6fc904 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:19.498111 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0e39224f-0766-4c44-930a-c903cf6fc904 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:19.504531 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1369/2731] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:27:19 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:27:19.505625 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ebffbc-fe51-46ae-bb40-f4182ce6dccf admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:19.515455 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e39224f-0766-4c44-930a-c903cf6fc904 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:19.518941 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ebffbc-fe51-46ae-bb40-f4182ce6dccf admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:19.520807 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b87a304-8a5a-4926-b639-376a09ec0824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:19.521979 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b87a304-8a5a-4926-b639-376a09ec0824 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:19.522199 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b87a304-8a5a-4926-b639-376a09ec0824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:19.522541 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b87a304-8a5a-4926-b639-376a09ec0824 admin admin] PATH_INFO: `/v3/users/9390ae25a8294693868b892ec92496a0` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:19.523896 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b87a304-8a5a-4926-b639-376a09ec0824 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9390ae25a8294693868b892ec92496a0)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:19.524410 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09446f9-fa34-4e91-b277-d3f3467aa195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:19.525325 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09446f9-fa34-4e91-b277-d3f3467aa195 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:19.525607 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09446f9-fa34-4e91-b277-d3f3467aa195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:19.525820 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b87a304-8a5a-4926-b639-376a09ec0824 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:19.525933 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09446f9-fa34-4e91-b277-d3f3467aa195 admin admin] PATH_INFO: `/v3/projects/a2b9fbff141540f0858c09f7ea1e80b9/users/df06c3b8aef249b984bc907b1753747c/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:19.549743 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09446f9-fa34-4e91-b277-d3f3467aa195 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2b9fbff141540f0858c09f7ea1e80b9, user_id=df06c3b8aef249b984bc907b1753747c, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:19.552660 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09446f9-fa34-4e91-b277-d3f3467aa195 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:19.567151 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1370/2732] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:27:19 2022] PUT /identity/v3/projects/a2b9fbff141540f0858c09f7ea1e80b9/users/df06c3b8aef249b984bc907b1753747c/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:27:19.567739 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09446f9-fa34-4e91-b277-d3f3467aa195 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:19.576335 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3b87a304-8a5a-4926-b639-376a09ec0824 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9390ae25a8294693868b892ec92496a0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:19.576999 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3b87a304-8a5a-4926-b639-376a09ec0824 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9390ae25a8294693868b892ec92496a0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:19.584949 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3b87a304-8a5a-4926-b639-376a09ec0824 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9390ae25a8294693868b892ec92496a0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:19.597211 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3b87a304-8a5a-4926-b639-376a09ec0824 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9390ae25a8294693868b892ec92496a0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:19.597894 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3b87a304-8a5a-4926-b639-376a09ec0824 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9390ae25a8294693868b892ec92496a0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:19.614295 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1363/2733] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:27:19 2022] DELETE /identity/v3/users/9390ae25a8294693868b892ec92496a0 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:27:19.616760 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034893a2-d418-47b7-8cd0-2804f7c0469f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:19.617081 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034893a2-d418-47b7-8cd0-2804f7c0469f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:19.617363 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034893a2-d418-47b7-8cd0-2804f7c0469f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:19.636893 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-034893a2-d418-47b7-8cd0-2804f7c0469f None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:27:19.638249 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-034893a2-d418-47b7-8cd0-2804f7c0469f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:19.639136 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09446f9-fa34-4e91-b277-d3f3467aa195 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:19.642273 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:27:19.660033 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:27:19.699088 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ce4a7c-2189-4e8b-88eb-5c5269202840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:19.699978 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce4a7c-2189-4e8b-88eb-5c5269202840 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:19.700262 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce4a7c-2189-4e8b-88eb-5c5269202840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:19.700676 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce4a7c-2189-4e8b-88eb-5c5269202840 admin admin] PATH_INFO: `/v3/projects/851c236d5b4f448bac8ff667ec27b98b/users/bfcf92ec9a854d14a4c85366810cc0e7/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:19.703129 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1364/2734] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:27:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:27:19.704438 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b93e78-6a6b-4356-8e33-ec44e346d86d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:19.704734 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b93e78-6a6b-4356-8e33-ec44e346d86d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:19.705020 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b93e78-6a6b-4356-8e33-ec44e346d86d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:19.716224 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ce4a7c-2189-4e8b-88eb-5c5269202840 admin admin] RBAC: Authorizing `identity:create_grant(project_id=851c236d5b4f448bac8ff667ec27b98b, user_id=bfcf92ec9a854d14a4c85366810cc0e7, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:19.718940 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ce4a7c-2189-4e8b-88eb-5c5269202840 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:19.723337 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b1b93e78-6a6b-4356-8e33-ec44e346d86d None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:27:19.724693 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b1b93e78-6a6b-4356-8e33-ec44e346d86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:19.732795 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1371/2735] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:27:19 2022] PUT /identity/v3/projects/851c236d5b4f448bac8ff667ec27b98b/users/bfcf92ec9a854d14a4c85366810cc0e7/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:27:19.733658 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ce4a7c-2189-4e8b-88eb-5c5269202840 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:19.735236 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-12ce4a7c-2189-4e8b-88eb-5c5269202840 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:19.793983 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1365/2736] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:27:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:27:19.795604 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c21f116-8fd8-4017-a059-77bd56d1ddf4 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:19.795894 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c21f116-8fd8-4017-a059-77bd56d1ddf4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:19.796172 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c21f116-8fd8-4017-a059-77bd56d1ddf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:19.800152 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ce4a7c-2189-4e8b-88eb-5c5269202840 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:19.805076 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24347cf7-48db-408b-850f-c0c239be002a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:19.805786 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24347cf7-48db-408b-850f-c0c239be002a service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:19.806000 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24347cf7-48db-408b-850f-c0c239be002a service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:19.806279 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24347cf7-48db-408b-850f-c0c239be002a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:19.816588 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24347cf7-48db-408b-850f-c0c239be002a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:19.818069 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24347cf7-48db-408b-850f-c0c239be002a service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:19.825392 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8c21f116-8fd8-4017-a059-77bd56d1ddf4 None None] MFA Rules not processed for user `bfcf92ec9a854d14a4c85366810cc0e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:27:19.826870 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8c21f116-8fd8-4017-a059-77bd56d1ddf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:19.828221 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1372/2737] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:27:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:27:19.829194 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24347cf7-48db-408b-850f-c0c239be002a service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:19.839517 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24347cf7-48db-408b-850f-c0c239be002a service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:19.843722 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c152c-4b4e-4d3d-a346-1e163314e7a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:19.844708 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c152c-4b4e-4d3d-a346-1e163314e7a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:19.845001 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c152c-4b4e-4d3d-a346-1e163314e7a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:19.845795 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c152c-4b4e-4d3d-a346-1e163314e7a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:19.857917 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1c152c-4b4e-4d3d-a346-1e163314e7a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:19.859628 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1c152c-4b4e-4d3d-a346-1e163314e7a2 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:19.871412 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1373/2738] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:27:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:27:19.900884 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1366/2739] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:27:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:27:19.912380 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c152c-4b4e-4d3d-a346-1e163314e7a2 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:19.914609 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-bb1c152c-4b4e-4d3d-a346-1e163314e7a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:19.977606 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c152c-4b4e-4d3d-a346-1e163314e7a2 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:19.981503 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c131e2-b957-437b-9768-dfd64eeb521f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:19.982158 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c131e2-b957-437b-9768-dfd64eeb521f service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:19.982446 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c131e2-b957-437b-9768-dfd64eeb521f service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:19.982725 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c131e2-b957-437b-9768-dfd64eeb521f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:19.992129 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c131e2-b957-437b-9768-dfd64eeb521f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:19.993639 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c131e2-b957-437b-9768-dfd64eeb521f service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:20.003418 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1374/2740] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:27:19 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:27:20.024533 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c21f116-8fd8-4017-a059-77bd56d1ddf4 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:20.026655 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8c21f116-8fd8-4017-a059-77bd56d1ddf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:20.042028 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c21f116-8fd8-4017-a059-77bd56d1ddf4 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:20.047682 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c84ce9a-ab89-4e94-b86f-60223fd8a0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:20.048680 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c84ce9a-ab89-4e94-b86f-60223fd8a0d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:20.049883 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c84ce9a-ab89-4e94-b86f-60223fd8a0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:20.050182 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c84ce9a-ab89-4e94-b86f-60223fd8a0d8 admin admin] PATH_INFO: `/v3/users/bfcf92ec9a854d14a4c85366810cc0e7` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:20.052023 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c84ce9a-ab89-4e94-b86f-60223fd8a0d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfcf92ec9a854d14a4c85366810cc0e7)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:20.054786 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c84ce9a-ab89-4e94-b86f-60223fd8a0d8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:20.097633 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:27:20.120906 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1c84ce9a-ab89-4e94-b86f-60223fd8a0d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfcf92ec9a854d14a4c85366810cc0e7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:20.121635 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1c84ce9a-ab89-4e94-b86f-60223fd8a0d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfcf92ec9a854d14a4c85366810cc0e7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:20.131114 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1c84ce9a-ab89-4e94-b86f-60223fd8a0d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfcf92ec9a854d14a4c85366810cc0e7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:20.150488 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1c84ce9a-ab89-4e94-b86f-60223fd8a0d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfcf92ec9a854d14a4c85366810cc0e7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:20.151580 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1c84ce9a-ab89-4e94-b86f-60223fd8a0d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfcf92ec9a854d14a4c85366810cc0e7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:20.170230 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1367/2741] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:27:20 2022] DELETE /identity/v3/users/bfcf92ec9a854d14a4c85366810cc0e7 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:27:20.174910 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bdb066-f811-4dca-95ed-bfdffcb957cd None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:20.175400 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bdb066-f811-4dca-95ed-bfdffcb957cd None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:20.175508 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bdb066-f811-4dca-95ed-bfdffcb957cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:20.196055 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-e2bdb066-f811-4dca-95ed-bfdffcb957cd None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:27:20.197475 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e2bdb066-f811-4dca-95ed-bfdffcb957cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:20.239474 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:27:20.255400 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c84ce9a-ab89-4e94-b86f-60223fd8a0d8 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:20.258136 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1c84ce9a-ab89-4e94-b86f-60223fd8a0d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:20.280851 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1375/2742] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:27:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:27:20.297669 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bdb066-f811-4dca-95ed-bfdffcb957cd None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:20.299732 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e2bdb066-f811-4dca-95ed-bfdffcb957cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:20.360598 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c84ce9a-ab89-4e94-b86f-60223fd8a0d8 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:20.375516 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bdb066-f811-4dca-95ed-bfdffcb957cd None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:20.375830 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ee1758-9b4a-4031-b577-895ee6eb0354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:20.376714 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee1758-9b4a-4031-b577-895ee6eb0354 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:20.377000 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee1758-9b4a-4031-b577-895ee6eb0354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:20.377304 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee1758-9b4a-4031-b577-895ee6eb0354 admin admin] PATH_INFO: `/v3/projects/21835652c80b4c719b04bb090eba6cd0` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:20.378673 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ee1758-9b4a-4031-b577-895ee6eb0354 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21835652c80b4c719b04bb090eba6cd0)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:20.379857 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525483ec-2e01-4e9e-984a-f7eb37b88e0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:20.380498 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ee1758-9b4a-4031-b577-895ee6eb0354 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:20.380637 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525483ec-2e01-4e9e-984a-f7eb37b88e0f service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:20.382063 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525483ec-2e01-4e9e-984a-f7eb37b88e0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:20.382063 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525483ec-2e01-4e9e-984a-f7eb37b88e0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:20.405876 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525483ec-2e01-4e9e-984a-f7eb37b88e0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:20.414689 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525483ec-2e01-4e9e-984a-f7eb37b88e0f service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:20.429911 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1376/2743] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:27:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:27:20.447742 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-33ee1758-9b4a-4031-b577-895ee6eb0354 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21835652c80b4c719b04bb090eba6cd0', 'request_id': 'req-33ee1758-9b4a-4031-b577-895ee6eb0354'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:20.453667 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-33ee1758-9b4a-4031-b577-895ee6eb0354 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21835652c80b4c719b04bb090eba6cd0', 'request_id': 'req-33ee1758-9b4a-4031-b577-895ee6eb0354'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:20.462412 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-33ee1758-9b4a-4031-b577-895ee6eb0354 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21835652c80b4c719b04bb090eba6cd0', 'request_id': 'req-33ee1758-9b4a-4031-b577-895ee6eb0354'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:20.477063 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-33ee1758-9b4a-4031-b577-895ee6eb0354 admin admin] The token cache is being invalidate because project 21835652c80b4c719b04bb090eba6cd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:27:20.477377 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-33ee1758-9b4a-4031-b577-895ee6eb0354 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:20.478089 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-33ee1758-9b4a-4031-b577-895ee6eb0354 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:20.479697 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1368/2744] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:27:20 2022] DELETE /identity/v3/projects/21835652c80b4c719b04bb090eba6cd0 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:27:20.484699 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee7be9b-aabb-4d45-99ec-d8583ddf25a9 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:20.484988 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee7be9b-aabb-4d45-99ec-d8583ddf25a9 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:20.485306 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee7be9b-aabb-4d45-99ec-d8583ddf25a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:20.503840 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-3ee7be9b-aabb-4d45-99ec-d8583ddf25a9 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:27:20.505073 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-3ee7be9b-aabb-4d45-99ec-d8583ddf25a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:20.574606 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1377/2745] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:27:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:27:20.582057 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc014f3-5df2-49b8-89d1-44a0bee5f07b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:20.582361 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc014f3-5df2-49b8-89d1-44a0bee5f07b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:20.582692 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc014f3-5df2-49b8-89d1-44a0bee5f07b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:20.607832 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8fc014f3-5df2-49b8-89d1-44a0bee5f07b None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:27:20.609184 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8fc014f3-5df2-49b8-89d1-44a0bee5f07b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:20.618176 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1369/2746] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:27:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:27:20.622377 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee7be9b-aabb-4d45-99ec-d8583ddf25a9 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:20.633808 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee7be9b-aabb-4d45-99ec-d8583ddf25a9 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:20.638947 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d293ba27-0abd-4d52-8122-8470d21616b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:20.639701 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293ba27-0abd-4d52-8122-8470d21616b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:20.639914 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293ba27-0abd-4d52-8122-8470d21616b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:20.640110 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293ba27-0abd-4d52-8122-8470d21616b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:20.640845 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d293ba27-0abd-4d52-8122-8470d21616b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:27:20.641191 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d293ba27-0abd-4d52-8122-8470d21616b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:20.642797 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d293ba27-0abd-4d52-8122-8470d21616b5 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:20.658052 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1378/2747] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:27:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:27:20.661684 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc014f3-5df2-49b8-89d1-44a0bee5f07b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:20.671396 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc014f3-5df2-49b8-89d1-44a0bee5f07b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:20.676473 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8775b40a-9e9f-4fe4-8395-39f61221abb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:20.677181 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8775b40a-9e9f-4fe4-8395-39f61221abb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:20.677463 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8775b40a-9e9f-4fe4-8395-39f61221abb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:20.677738 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8775b40a-9e9f-4fe4-8395-39f61221abb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:20.679680 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8775b40a-9e9f-4fe4-8395-39f61221abb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:20.681739 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8775b40a-9e9f-4fe4-8395-39f61221abb2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:20.712410 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1370/2748] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:27:20 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:27:20.713691 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d293ba27-0abd-4d52-8122-8470d21616b5 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:20.729801 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d293ba27-0abd-4d52-8122-8470d21616b5 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:20.821175 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6076fd1c-575a-4940-ac3e-035ca4b164fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:20.821925 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6076fd1c-575a-4940-ac3e-035ca4b164fe admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:20.822256 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6076fd1c-575a-4940-ac3e-035ca4b164fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:20.822522 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6076fd1c-575a-4940-ac3e-035ca4b164fe admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:20.823220 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6076fd1c-575a-4940-ac3e-035ca4b164fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:20.825174 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6076fd1c-575a-4940-ac3e-035ca4b164fe admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:20.846851 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:27:20.855981 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1379/2749] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:27:20 2022] POST /identity/v3/users => generated 283 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:27:20.860333 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8775b40a-9e9f-4fe4-8395-39f61221abb2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:20.874699 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8775b40a-9e9f-4fe4-8395-39f61221abb2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:20.882443 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794dc5a5-2cfb-4a7c-8360-2dec24558046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:20.882818 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794dc5a5-2cfb-4a7c-8360-2dec24558046 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:20.883065 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794dc5a5-2cfb-4a7c-8360-2dec24558046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:20.883409 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794dc5a5-2cfb-4a7c-8360-2dec24558046 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:20.884948 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794dc5a5-2cfb-4a7c-8360-2dec24558046 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:20.886924 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794dc5a5-2cfb-4a7c-8360-2dec24558046 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:20.891470 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6076fd1c-575a-4940-ac3e-035ca4b164fe admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:20.893471 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-6076fd1c-575a-4940-ac3e-035ca4b164fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:20.904146 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1371/2750] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:27:20 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:27:20.913648 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794dc5a5-2cfb-4a7c-8360-2dec24558046 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:20.916069 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6076fd1c-575a-4940-ac3e-035ca4b164fe admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:20.920414 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:27:20.925949 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bb4786-dfa1-4f2e-b85f-54ed69f02821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:20.926816 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794dc5a5-2cfb-4a7c-8360-2dec24558046 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:20.927309 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bb4786-dfa1-4f2e-b85f-54ed69f02821 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:20.927604 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bb4786-dfa1-4f2e-b85f-54ed69f02821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:20.927890 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bb4786-dfa1-4f2e-b85f-54ed69f02821 admin admin] PATH_INFO: `/v3/projects/851c236d5b4f448bac8ff667ec27b98b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:20.929791 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bb4786-dfa1-4f2e-b85f-54ed69f02821 admin admin] RBAC: Authorizing `identity:delete_project(project_id=851c236d5b4f448bac8ff667ec27b98b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:20.932323 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bb4786-dfa1-4f2e-b85f-54ed69f02821 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:20.935834 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51190e07-021b-4ffe-8fb5-7986e98eacc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:20.937434 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51190e07-021b-4ffe-8fb5-7986e98eacc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:20.937830 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51190e07-021b-4ffe-8fb5-7986e98eacc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:20.938120 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51190e07-021b-4ffe-8fb5-7986e98eacc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:20.938910 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51190e07-021b-4ffe-8fb5-7986e98eacc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:20.941473 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51190e07-021b-4ffe-8fb5-7986e98eacc1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:20.958922 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1372/2751] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:27:20 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:27:20.963777 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51190e07-021b-4ffe-8fb5-7986e98eacc1 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:20.982735 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51190e07-021b-4ffe-8fb5-7986e98eacc1 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:20.989862 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c0f46c-37c9-4b8f-8761-0b2342c31624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:20.992071 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c0f46c-37c9-4b8f-8761-0b2342c31624 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:20.992364 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c0f46c-37c9-4b8f-8761-0b2342c31624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:20.992653 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c0f46c-37c9-4b8f-8761-0b2342c31624 admin admin] PATH_INFO: `/v3/projects/e1b430045396418d97cbc6b7dc9de645/users/aebfbcf03e2046ad918f84a6ee457dc4/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:21.013728 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c0f46c-37c9-4b8f-8761-0b2342c31624 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1b430045396418d97cbc6b7dc9de645, user_id=aebfbcf03e2046ad918f84a6ee457dc4, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:21.017615 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c3bb4786-dfa1-4f2e-b85f-54ed69f02821 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '851c236d5b4f448bac8ff667ec27b98b', 'request_id': 'req-c3bb4786-dfa1-4f2e-b85f-54ed69f02821'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:21.018969 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c0f46c-37c9-4b8f-8761-0b2342c31624 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:21.024850 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c3bb4786-dfa1-4f2e-b85f-54ed69f02821 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '851c236d5b4f448bac8ff667ec27b98b', 'request_id': 'req-c3bb4786-dfa1-4f2e-b85f-54ed69f02821'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:21.035821 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c3bb4786-dfa1-4f2e-b85f-54ed69f02821 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '851c236d5b4f448bac8ff667ec27b98b', 'request_id': 'req-c3bb4786-dfa1-4f2e-b85f-54ed69f02821'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:21.039455 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1373/2752] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:27:20 2022] PUT /identity/v3/projects/e1b430045396418d97cbc6b7dc9de645/users/aebfbcf03e2046ad918f84a6ee457dc4/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:27:21.050651 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf100a-3bec-4963-8600-b74bdc6e6ff9 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:21.050948 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf100a-3bec-4963-8600-b74bdc6e6ff9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:21.051234 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf100a-3bec-4963-8600-b74bdc6e6ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:21.066859 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c3bb4786-dfa1-4f2e-b85f-54ed69f02821 admin admin] The token cache is being invalidate because project 851c236d5b4f448bac8ff667ec27b98b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:27:21.067152 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c3bb4786-dfa1-4f2e-b85f-54ed69f02821 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:21.067831 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c3bb4786-dfa1-4f2e-b85f-54ed69f02821 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:21.070564 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1380/2753] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:27:20 2022] DELETE /identity/v3/projects/851c236d5b4f448bac8ff667ec27b98b => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:27:21.076429 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2855745-9aaa-4dd1-a42d-ab87e0d8746c None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:21.076892 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2855745-9aaa-4dd1-a42d-ab87e0d8746c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:21.077268 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2855745-9aaa-4dd1-a42d-ab87e0d8746c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:21.093518 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-cfdf100a-3bec-4963-8600-b74bdc6e6ff9 None None] MFA Rules not processed for user `aebfbcf03e2046ad918f84a6ee457dc4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:27:21.095028 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cfdf100a-3bec-4963-8600-b74bdc6e6ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:21.100694 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-a2855745-9aaa-4dd1-a42d-ab87e0d8746c None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:27:21.102798 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a2855745-9aaa-4dd1-a42d-ab87e0d8746c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:21.166730 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1374/2754] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:27:21 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:27:21.181257 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdf100a-3bec-4963-8600-b74bdc6e6ff9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:21.183239 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cfdf100a-3bec-4963-8600-b74bdc6e6ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:21.189227 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1381/2755] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:27:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:27:21.193341 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632726f5-46d1-4759-ba46-8008f84867d5 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:21.194119 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632726f5-46d1-4759-ba46-8008f84867d5 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:21.194119 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632726f5-46d1-4759-ba46-8008f84867d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:21.225975 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-632726f5-46d1-4759-ba46-8008f84867d5 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:27:21.227685 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-632726f5-46d1-4759-ba46-8008f84867d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:21.245230 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1382/2756] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:27:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:27:21.247973 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632726f5-46d1-4759-ba46-8008f84867d5 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:21.261444 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632726f5-46d1-4759-ba46-8008f84867d5 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:21.267325 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43de0fd-0398-4460-8fe8-cd12cf396855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:21.268357 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43de0fd-0398-4460-8fe8-cd12cf396855 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:21.269106 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43de0fd-0398-4460-8fe8-cd12cf396855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:21.269630 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43de0fd-0398-4460-8fe8-cd12cf396855 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:21.270616 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43de0fd-0398-4460-8fe8-cd12cf396855 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:27:21.271208 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43de0fd-0398-4460-8fe8-cd12cf396855 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:21.273754 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43de0fd-0398-4460-8fe8-cd12cf396855 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:21.279922 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdf100a-3bec-4963-8600-b74bdc6e6ff9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:21.284342 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b009aa55-b024-47f8-a5af-758ee6e42f85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:21.285004 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b009aa55-b024-47f8-a5af-758ee6e42f85 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:21.285304 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b009aa55-b024-47f8-a5af-758ee6e42f85 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:21.285589 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b009aa55-b024-47f8-a5af-758ee6e42f85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:21.299703 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1383/2757] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:27:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:27:21.300797 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43de0fd-0398-4460-8fe8-cd12cf396855 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:21.307623 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b009aa55-b024-47f8-a5af-758ee6e42f85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:21.310267 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b009aa55-b024-47f8-a5af-758ee6e42f85 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:21.316689 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43de0fd-0398-4460-8fe8-cd12cf396855 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:21.326740 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1375/2758] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:27:21 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:27:21.333793 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6e2df8-d768-4262-8787-e8348f24091e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:21.334697 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e2df8-d768-4262-8787-e8348f24091e admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:21.335324 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e2df8-d768-4262-8787-e8348f24091e admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:21.335481 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e2df8-d768-4262-8787-e8348f24091e admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:21.336889 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6e2df8-d768-4262-8787-e8348f24091e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:21.354567 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6e2df8-d768-4262-8787-e8348f24091e admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:21.422262 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b009aa55-b024-47f8-a5af-758ee6e42f85 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:21.422442 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1384/2759] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:27:21 2022] POST /identity/v3/projects => generated 385 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:27:21.442134 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b009aa55-b024-47f8-a5af-758ee6e42f85 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:21.536488 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b01e4c-246c-4b5b-9102-cb49f789bfaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:21.538477 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b01e4c-246c-4b5b-9102-cb49f789bfaf admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:21.538983 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b01e4c-246c-4b5b-9102-cb49f789bfaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:21.539278 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b01e4c-246c-4b5b-9102-cb49f789bfaf admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:21.540106 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b01e4c-246c-4b5b-9102-cb49f789bfaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:21.542517 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b01e4c-246c-4b5b-9102-cb49f789bfaf admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:21.550935 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:27:21.573082 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1376/2760] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:27:21 2022] POST /identity/v3/users => generated 293 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:27:21.576461 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6e2df8-d768-4262-8787-e8348f24091e admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:21.593708 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6e2df8-d768-4262-8787-e8348f24091e admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:21.613554 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed434ebf-e049-439e-86e4-de83106bc6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:21.614072 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed434ebf-e049-439e-86e4-de83106bc6db admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:21.614391 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed434ebf-e049-439e-86e4-de83106bc6db admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:21.614683 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed434ebf-e049-439e-86e4-de83106bc6db admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:21.615858 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed434ebf-e049-439e-86e4-de83106bc6db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:21.617756 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed434ebf-e049-439e-86e4-de83106bc6db admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:21.634041 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1385/2761] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:27:21.634910 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b01e4c-246c-4b5b-9102-cb49f789bfaf admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:21.645787 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b01e4c-246c-4b5b-9102-cb49f789bfaf admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:21.655614 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb3271-e9a6-47ef-945b-79120674c8d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:21.656451 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb3271-e9a6-47ef-945b-79120674c8d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:21.656739 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb3271-e9a6-47ef-945b-79120674c8d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:21.657020 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb3271-e9a6-47ef-945b-79120674c8d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:21.657789 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afb3271-e9a6-47ef-945b-79120674c8d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:21.668690 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afb3271-e9a6-47ef-945b-79120674c8d0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:21.689097 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1377/2762] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:27:21.690953 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed434ebf-e049-439e-86e4-de83106bc6db admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:21.703108 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed434ebf-e049-439e-86e4-de83106bc6db admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:21.709872 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89e1451-752f-44f3-915d-e26263880357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:21.711392 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e1451-752f-44f3-915d-e26263880357 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:21.711920 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e1451-752f-44f3-915d-e26263880357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:21.712535 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e1451-752f-44f3-915d-e26263880357 admin admin] PATH_INFO: `/v3/projects/33ece32c0c9e48d6b81a8f57d60f214c/users/fcc540ec055a41dca30c17ae29756e18/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:21.739515 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89e1451-752f-44f3-915d-e26263880357 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33ece32c0c9e48d6b81a8f57d60f214c, user_id=fcc540ec055a41dca30c17ae29756e18, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:21.745376 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89e1451-752f-44f3-915d-e26263880357 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:21.771543 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1386/2763] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:27:21 2022] PUT /identity/v3/projects/33ece32c0c9e48d6b81a8f57d60f214c/users/fcc540ec055a41dca30c17ae29756e18/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:27:21.785869 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a8ee20-e889-4d7f-b462-7aa56914dd7e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:21.786382 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a8ee20-e889-4d7f-b462-7aa56914dd7e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:21.786525 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a8ee20-e889-4d7f-b462-7aa56914dd7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:21.827418 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b6a8ee20-e889-4d7f-b462-7aa56914dd7e None None] MFA Rules not processed for user `fcc540ec055a41dca30c17ae29756e18`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:27:21.828699 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b6a8ee20-e889-4d7f-b462-7aa56914dd7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:21.945702 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1378/2764] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:27:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:27:21.949733 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89e1451-752f-44f3-915d-e26263880357 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:21.959920 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:27:22.034037 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89e1451-752f-44f3-915d-e26263880357 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:22.039099 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e06558b-c274-46b7-91e6-83aebb2124f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:22.039767 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06558b-c274-46b7-91e6-83aebb2124f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:22.040058 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06558b-c274-46b7-91e6-83aebb2124f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:22.040341 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06558b-c274-46b7-91e6-83aebb2124f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:22.041078 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e06558b-c274-46b7-91e6-83aebb2124f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:22.043008 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e06558b-c274-46b7-91e6-83aebb2124f3 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:22.073397 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1387/2765] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:27:21 2022] POST /identity/v3/projects => generated 385 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:27:22.077652 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a8ee20-e889-4d7f-b462-7aa56914dd7e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:22.089518 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a8ee20-e889-4d7f-b462-7aa56914dd7e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:22.153849 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabda2d0-1741-409f-9a70-94280c559dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:22.154532 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabda2d0-1741-409f-9a70-94280c559dfa admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:22.154818 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabda2d0-1741-409f-9a70-94280c559dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:22.155100 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabda2d0-1741-409f-9a70-94280c559dfa admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:22.155811 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabda2d0-1741-409f-9a70-94280c559dfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:22.157985 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabda2d0-1741-409f-9a70-94280c559dfa admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:22.180773 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1379/2766] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:27:22 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:27:22.184820 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e06558b-c274-46b7-91e6-83aebb2124f3 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:22.196118 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e06558b-c274-46b7-91e6-83aebb2124f3 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:22.201125 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108ffda0-dee6-4972-aee0-f02e04698ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:22.201927 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ffda0-dee6-4972-aee0-f02e04698ba4 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:22.202285 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ffda0-dee6-4972-aee0-f02e04698ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:22.202636 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ffda0-dee6-4972-aee0-f02e04698ba4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:22.203440 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108ffda0-dee6-4972-aee0-f02e04698ba4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:22.205163 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108ffda0-dee6-4972-aee0-f02e04698ba4 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:22.218012 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1388/2767] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:27:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:27:22.222460 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabda2d0-1741-409f-9a70-94280c559dfa admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:22.233983 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabda2d0-1741-409f-9a70-94280c559dfa admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:22.239377 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bf956c-9213-4c8b-a7e3-9916141015fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:22.240121 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf956c-9213-4c8b-a7e3-9916141015fa admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:22.240552 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf956c-9213-4c8b-a7e3-9916141015fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:22.240968 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf956c-9213-4c8b-a7e3-9916141015fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:22.241844 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bf956c-9213-4c8b-a7e3-9916141015fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:22.243542 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bf956c-9213-4c8b-a7e3-9916141015fa admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:22.257478 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1380/2768] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:27:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:27:22.258696 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108ffda0-dee6-4972-aee0-f02e04698ba4 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:22.268989 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108ffda0-dee6-4972-aee0-f02e04698ba4 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:22.274466 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82d4b44-abe0-410e-aed1-3380fec49b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:22.275421 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d4b44-abe0-410e-aed1-3380fec49b81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:22.275775 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d4b44-abe0-410e-aed1-3380fec49b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:22.276121 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d4b44-abe0-410e-aed1-3380fec49b81 admin admin] PATH_INFO: `/v3/projects/fdba18a4556547b39b35723ae54a1ef0/users/2999125ffbd64ef79ff6e97045d859e6/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:22.292122 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82d4b44-abe0-410e-aed1-3380fec49b81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdba18a4556547b39b35723ae54a1ef0, user_id=2999125ffbd64ef79ff6e97045d859e6, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:22.294920 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82d4b44-abe0-410e-aed1-3380fec49b81 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:22.308182 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1389/2769] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:27:22 2022] PUT /identity/v3/projects/fdba18a4556547b39b35723ae54a1ef0/users/2999125ffbd64ef79ff6e97045d859e6/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:27:22.318472 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc8ba90-fa94-4cf8-9881-35c6928fb680 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:22.318954 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc8ba90-fa94-4cf8-9881-35c6928fb680 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:22.319247 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc8ba90-fa94-4cf8-9881-35c6928fb680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:22.346755 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7cc8ba90-fa94-4cf8-9881-35c6928fb680 None None] MFA Rules not processed for user `2999125ffbd64ef79ff6e97045d859e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:27:22.347985 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7cc8ba90-fa94-4cf8-9881-35c6928fb680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:22.397723 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:27:22.438427 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1381/2770] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:27:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:27:22.443146 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82d4b44-abe0-410e-aed1-3380fec49b81 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:22.520191 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82d4b44-abe0-410e-aed1-3380fec49b81 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:22.526185 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300b390-9372-426b-a154-029283842480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:22.526995 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300b390-9372-426b-a154-029283842480 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:22.527388 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300b390-9372-426b-a154-029283842480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:22.527751 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300b390-9372-426b-a154-029283842480 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:22.528621 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c300b390-9372-426b-a154-029283842480 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:22.530594 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c300b390-9372-426b-a154-029283842480 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:22.560588 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1390/2771] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:27:22 2022] POST /identity/v3/projects => generated 385 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:27:22.564988 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc8ba90-fa94-4cf8-9881-35c6928fb680 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:22.580926 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc8ba90-fa94-4cf8-9881-35c6928fb680 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:22.645579 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d44016b-ecca-4ee7-a772-02dd24c7fb68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:22.646559 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d44016b-ecca-4ee7-a772-02dd24c7fb68 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:22.647490 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d44016b-ecca-4ee7-a772-02dd24c7fb68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:22.648266 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d44016b-ecca-4ee7-a772-02dd24c7fb68 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:22.649098 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d44016b-ecca-4ee7-a772-02dd24c7fb68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:22.650928 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d44016b-ecca-4ee7-a772-02dd24c7fb68 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:22.675532 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1382/2772] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:27:22 2022] POST /identity/v3/users => generated 299 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:27:22.681071 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300b390-9372-426b-a154-029283842480 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:22.694212 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300b390-9372-426b-a154-029283842480 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:22.700469 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeb4ccc-78e0-44f3-a36e-d1476cda64d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:22.701576 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb4ccc-78e0-44f3-a36e-d1476cda64d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:22.701822 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb4ccc-78e0-44f3-a36e-d1476cda64d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:22.702205 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb4ccc-78e0-44f3-a36e-d1476cda64d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:22.703256 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeb4ccc-78e0-44f3-a36e-d1476cda64d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:22.705385 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeb4ccc-78e0-44f3-a36e-d1476cda64d2 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:22.719208 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1391/2773] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:27:22 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:27:22.729687 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d44016b-ecca-4ee7-a772-02dd24c7fb68 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:22.741545 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d44016b-ecca-4ee7-a772-02dd24c7fb68 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:22.747453 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff7afb-8c42-4c76-8ce4-55b4da7d3ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:22.748542 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff7afb-8c42-4c76-8ce4-55b4da7d3ee5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:22.748904 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff7afb-8c42-4c76-8ce4-55b4da7d3ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:22.749321 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff7afb-8c42-4c76-8ce4-55b4da7d3ee5 admin admin] PATH_INFO: `/v3/projects/5ff88fefbf5c41f384872f427a9561d9/users/1d32d8f4131b4df2ad68b413115fbfc2/roles/f36e053b442540f08041ac7878760f65` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:22.770379 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ff7afb-8c42-4c76-8ce4-55b4da7d3ee5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ff88fefbf5c41f384872f427a9561d9, user_id=1d32d8f4131b4df2ad68b413115fbfc2, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:22.773490 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ff7afb-8c42-4c76-8ce4-55b4da7d3ee5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:22.795652 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1383/2774] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:27:22 2022] PUT /identity/v3/projects/5ff88fefbf5c41f384872f427a9561d9/users/1d32d8f4131b4df2ad68b413115fbfc2/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:27:22.802195 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeb4ccc-78e0-44f3-a36e-d1476cda64d2 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:22.818664 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeb4ccc-78e0-44f3-a36e-d1476cda64d2 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:22.858049 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:27:22.915994 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53187ea0-1522-486d-9f0c-c099468a92a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:22.916696 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53187ea0-1522-486d-9f0c-c099468a92a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:22.917083 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53187ea0-1522-486d-9f0c-c099468a92a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:22.917576 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53187ea0-1522-486d-9f0c-c099468a92a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:22.918529 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53187ea0-1522-486d-9f0c-c099468a92a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:22.920431 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53187ea0-1522-486d-9f0c-c099468a92a8 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:22.935915 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1392/2775] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:27:22 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:27:22.937379 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff7afb-8c42-4c76-8ce4-55b4da7d3ee5 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:22.952858 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff7afb-8c42-4c76-8ce4-55b4da7d3ee5 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:22.967666 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f6a69d-4536-41c2-8fae-b71a01473673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:22.968588 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f6a69d-4536-41c2-8fae-b71a01473673 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:22.968875 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f6a69d-4536-41c2-8fae-b71a01473673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:22.969185 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f6a69d-4536-41c2-8fae-b71a01473673 admin admin] PATH_INFO: `/v3/projects/5ff88fefbf5c41f384872f427a9561d9/users/1d32d8f4131b4df2ad68b413115fbfc2/roles/f36e053b442540f08041ac7878760f65` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:22.981266 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f6a69d-4536-41c2-8fae-b71a01473673 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ff88fefbf5c41f384872f427a9561d9, user_id=1d32d8f4131b4df2ad68b413115fbfc2, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:22.983932 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f6a69d-4536-41c2-8fae-b71a01473673 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:23.020984 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1384/2776] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:27:22 2022] PUT /identity/v3/projects/5ff88fefbf5c41f384872f427a9561d9/users/1d32d8f4131b4df2ad68b413115fbfc2/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:27:23.099931 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba939da3-cdee-46c4-8c24-b9adf573942f None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:23.100798 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba939da3-cdee-46c4-8c24-b9adf573942f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:23.101121 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba939da3-cdee-46c4-8c24-b9adf573942f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:23.174226 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-ba939da3-cdee-46c4-8c24-b9adf573942f None None] MFA Rules not processed for user `1d32d8f4131b4df2ad68b413115fbfc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:27:23.174226 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ba939da3-cdee-46c4-8c24-b9adf573942f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:23.276965 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1393/2777] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:27:23 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:27:23.277685 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f6a69d-4536-41c2-8fae-b71a01473673 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:23.279828 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-69f6a69d-4536-41c2-8fae-b71a01473673 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:23.363634 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f6a69d-4536-41c2-8fae-b71a01473673 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:23.370141 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232bbcf6-bf89-4339-8dbe-d72bf00d7212 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:23.371413 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232bbcf6-bf89-4339-8dbe-d72bf00d7212 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:23.371744 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232bbcf6-bf89-4339-8dbe-d72bf00d7212 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:23.371982 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232bbcf6-bf89-4339-8dbe-d72bf00d7212 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:23.438123 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:27:23.482935 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232bbcf6-bf89-4339-8dbe-d72bf00d7212 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:23.487926 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232bbcf6-bf89-4339-8dbe-d72bf00d7212 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:23.511152 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1385/2778] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:27:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:27:25.094671 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefce33a-5daa-4d66-804b-1ecd1e889844 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:25.095162 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefce33a-5daa-4d66-804b-1ecd1e889844 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:25.095389 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefce33a-5daa-4d66-804b-1ecd1e889844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:25.129217 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-eefce33a-5daa-4d66-804b-1ecd1e889844 None None] MFA Rules not processed for user `df06c3b8aef249b984bc907b1753747c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:27:25.130474 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-eefce33a-5daa-4d66-804b-1ecd1e889844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:25.216888 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1394/2779] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:27:25 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:27:25.221806 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232bbcf6-bf89-4339-8dbe-d72bf00d7212 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:25.223701 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-232bbcf6-bf89-4339-8dbe-d72bf00d7212 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:25.301135 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232bbcf6-bf89-4339-8dbe-d72bf00d7212 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:25.306947 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c33e46b-e161-4a19-83f6-dea9ca96aa95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:25.307757 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c33e46b-e161-4a19-83f6-dea9ca96aa95 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:25.308348 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c33e46b-e161-4a19-83f6-dea9ca96aa95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:25.308452 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c33e46b-e161-4a19-83f6-dea9ca96aa95 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:25.309267 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c33e46b-e161-4a19-83f6-dea9ca96aa95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:25.311834 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c33e46b-e161-4a19-83f6-dea9ca96aa95 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:25.317002 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefce33a-5daa-4d66-804b-1ecd1e889844 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:25.332061 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefce33a-5daa-4d66-804b-1ecd1e889844 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:25.338026 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18fbae-b4fb-451a-a410-899d95e561de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:25.338785 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18fbae-b4fb-451a-a410-899d95e561de service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:25.339149 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18fbae-b4fb-451a-a410-899d95e561de service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:25.339524 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18fbae-b4fb-451a-a410-899d95e561de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:25.356877 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1386/2780] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:27:25 2022] POST /identity/v3/projects => generated 373 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:27:25.360733 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c33e46b-e161-4a19-83f6-dea9ca96aa95 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:25.404877 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:27:25.427932 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed18fbae-b4fb-451a-a410-899d95e561de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:25.430147 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed18fbae-b4fb-451a-a410-899d95e561de service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:25.439843 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1395/2781] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:27:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:27:25.440617 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18fbae-b4fb-451a-a410-899d95e561de service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:25.442142 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ed18fbae-b4fb-451a-a410-899d95e561de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:25.447460 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c33e46b-e161-4a19-83f6-dea9ca96aa95 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:25.453548 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac11c86-646f-4383-b4f4-3aa921c2b29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:25.454191 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac11c86-646f-4383-b4f4-3aa921c2b29f admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:25.454504 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac11c86-646f-4383-b4f4-3aa921c2b29f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:25.454821 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac11c86-646f-4383-b4f4-3aa921c2b29f admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:25.455473 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac11c86-646f-4383-b4f4-3aa921c2b29f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:25.458481 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac11c86-646f-4383-b4f4-3aa921c2b29f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:25.462413 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18fbae-b4fb-451a-a410-899d95e561de service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:25.471256 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb48f1bf-e0ae-4dbd-97f4-e8e09083f89b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:25.472203 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb48f1bf-e0ae-4dbd-97f4-e8e09083f89b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:25.472484 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb48f1bf-e0ae-4dbd-97f4-e8e09083f89b admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:25.472766 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb48f1bf-e0ae-4dbd-97f4-e8e09083f89b admin admin] PATH_INFO: `/v3/users/dc04b222c39248bebe8506dbf067efba` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:25.474123 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb48f1bf-e0ae-4dbd-97f4-e8e09083f89b admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc04b222c39248bebe8506dbf067efba)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:25.477468 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb48f1bf-e0ae-4dbd-97f4-e8e09083f89b admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:25.488527 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1387/2782] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:27:25 2022] POST /identity/v3/users => generated 293 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:27:25.489474 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac11c86-646f-4383-b4f4-3aa921c2b29f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:25.500622 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac11c86-646f-4383-b4f4-3aa921c2b29f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:25.514995 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea44631a-14a2-45a6-90ee-9d770a7864a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:25.516735 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea44631a-14a2-45a6-90ee-9d770a7864a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:25.517027 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea44631a-14a2-45a6-90ee-9d770a7864a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:25.517333 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea44631a-14a2-45a6-90ee-9d770a7864a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:25.518280 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea44631a-14a2-45a6-90ee-9d770a7864a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:25.520556 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea44631a-14a2-45a6-90ee-9d770a7864a4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:25.535678 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1388/2783] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:27:25 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:27:25.540637 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea44631a-14a2-45a6-90ee-9d770a7864a4 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:25.553997 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea44631a-14a2-45a6-90ee-9d770a7864a4 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:25.560947 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1445bd30-e314-4945-95cc-dfabf05f177a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:25.562549 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445bd30-e314-4945-95cc-dfabf05f177a admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:25.563507 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445bd30-e314-4945-95cc-dfabf05f177a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:25.563872 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445bd30-e314-4945-95cc-dfabf05f177a admin admin] PATH_INFO: `/v3/projects/5b984458174d4101bae3cff412ff7ef2/users/7fd3ab8ccfda463f9eed5641141e0c87/roles/f36e053b442540f08041ac7878760f65` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:25.567886 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-cb48f1bf-e0ae-4dbd-97f4-e8e09083f89b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc04b222c39248bebe8506dbf067efba'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:25.576329 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-cb48f1bf-e0ae-4dbd-97f4-e8e09083f89b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc04b222c39248bebe8506dbf067efba'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:25.577101 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-cb48f1bf-e0ae-4dbd-97f4-e8e09083f89b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc04b222c39248bebe8506dbf067efba'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:25.586669 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-cb48f1bf-e0ae-4dbd-97f4-e8e09083f89b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc04b222c39248bebe8506dbf067efba'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:25.590144 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1445bd30-e314-4945-95cc-dfabf05f177a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b984458174d4101bae3cff412ff7ef2, user_id=7fd3ab8ccfda463f9eed5641141e0c87, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:25.593235 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1445bd30-e314-4945-95cc-dfabf05f177a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:25.607157 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-cb48f1bf-e0ae-4dbd-97f4-e8e09083f89b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc04b222c39248bebe8506dbf067efba'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:25.614855 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1389/2784] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:27:25 2022] PUT /identity/v3/projects/5b984458174d4101bae3cff412ff7ef2/users/7fd3ab8ccfda463f9eed5641141e0c87/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:27:25.615073 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1396/2785] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:27:25 2022] DELETE /identity/v3/users/dc04b222c39248bebe8506dbf067efba => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:27:25.618255 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1445bd30-e314-4945-95cc-dfabf05f177a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:25.620139 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e3403-711e-4e60-b090-99470913e77f None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:25.620323 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1445bd30-e314-4945-95cc-dfabf05f177a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:25.620553 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e3403-711e-4e60-b090-99470913e77f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:25.620894 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e3403-711e-4e60-b090-99470913e77f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:25.646026 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-316e3403-711e-4e60-b090-99470913e77f None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:27:25.647612 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-316e3403-711e-4e60-b090-99470913e77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:25.655195 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:27:25.709753 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1445bd30-e314-4945-95cc-dfabf05f177a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:25.753828 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1397/2786] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:27:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:27:25.761347 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e3403-711e-4e60-b090-99470913e77f None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:25.763183 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-316e3403-711e-4e60-b090-99470913e77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:25.789747 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d4e0c1-974e-4716-91da-db3ea0a2c650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:25.790794 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4e0c1-974e-4716-91da-db3ea0a2c650 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:25.791374 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4e0c1-974e-4716-91da-db3ea0a2c650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:25.791840 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4e0c1-974e-4716-91da-db3ea0a2c650 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:25.792766 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d4e0c1-974e-4716-91da-db3ea0a2c650 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:25.794601 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d4e0c1-974e-4716-91da-db3ea0a2c650 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:25.808022 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1390/2787] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:27:25 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:27:25.812134 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d4e0c1-974e-4716-91da-db3ea0a2c650 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:25.825002 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d4e0c1-974e-4716-91da-db3ea0a2c650 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:25.830497 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3778994a-ae64-4e0d-8aef-88dc908828fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:25.831566 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3778994a-ae64-4e0d-8aef-88dc908828fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:25.831850 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3778994a-ae64-4e0d-8aef-88dc908828fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:25.832148 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3778994a-ae64-4e0d-8aef-88dc908828fd admin admin] PATH_INFO: `/v3/projects/5b984458174d4101bae3cff412ff7ef2/users/7fd3ab8ccfda463f9eed5641141e0c87/roles/f36e053b442540f08041ac7878760f65` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:25.834419 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3778994a-ae64-4e0d-8aef-88dc908828fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b984458174d4101bae3cff412ff7ef2, user_id=7fd3ab8ccfda463f9eed5641141e0c87, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:25.837567 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3778994a-ae64-4e0d-8aef-88dc908828fd admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:25.839982 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e3403-711e-4e60-b090-99470913e77f None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:25.845000 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a82c0c-110b-45f5-8690-c9db323f88ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:25.846003 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a82c0c-110b-45f5-8690-c9db323f88ac service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:25.846298 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a82c0c-110b-45f5-8690-c9db323f88ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:25.846591 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a82c0c-110b-45f5-8690-c9db323f88ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:25.851286 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1391/2788] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:27:25 2022] PUT /identity/v3/projects/5b984458174d4101bae3cff412ff7ef2/users/7fd3ab8ccfda463f9eed5641141e0c87/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:27:25.856208 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:27:25.858998 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a82c0c-110b-45f5-8690-c9db323f88ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:25.860707 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a82c0c-110b-45f5-8690-c9db323f88ac service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:25.875073 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1398/2789] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:27:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:27:26.285619 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3778994a-ae64-4e0d-8aef-88dc908828fd admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:26.288366 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3778994a-ae64-4e0d-8aef-88dc908828fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:26.338700 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a82c0c-110b-45f5-8690-c9db323f88ac service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:26.340790 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a4a82c0c-110b-45f5-8690-c9db323f88ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:26.358825 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a82c0c-110b-45f5-8690-c9db323f88ac service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:26.436104 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3778994a-ae64-4e0d-8aef-88dc908828fd admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:26.439886 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3743e67-5a25-4dcf-b4d7-a3561863cb39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:26.440547 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3743e67-5a25-4dcf-b4d7-a3561863cb39 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:26.440852 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3743e67-5a25-4dcf-b4d7-a3561863cb39 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:26.441329 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3743e67-5a25-4dcf-b4d7-a3561863cb39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:26.443204 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d3743e67-5a25-4dcf-b4d7-a3561863cb39 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:26.445935 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b76eaac-c67d-4fe6-a400-05beda6df99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:26.447754 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b76eaac-c67d-4fe6-a400-05beda6df99c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:26.448536 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b76eaac-c67d-4fe6-a400-05beda6df99c admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:26.448536 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b76eaac-c67d-4fe6-a400-05beda6df99c admin admin] PATH_INFO: `/v3/projects/fd760a4c64954833885e2b8d400100f4` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:26.451408 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b76eaac-c67d-4fe6-a400-05beda6df99c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd760a4c64954833885e2b8d400100f4)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:26.452076 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b76eaac-c67d-4fe6-a400-05beda6df99c admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:26.511501 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-7b76eaac-c67d-4fe6-a400-05beda6df99c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd760a4c64954833885e2b8d400100f4', 'request_id': 'req-7b76eaac-c67d-4fe6-a400-05beda6df99c'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:26.519720 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-7b76eaac-c67d-4fe6-a400-05beda6df99c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd760a4c64954833885e2b8d400100f4', 'request_id': 'req-7b76eaac-c67d-4fe6-a400-05beda6df99c'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:26.530998 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-7b76eaac-c67d-4fe6-a400-05beda6df99c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd760a4c64954833885e2b8d400100f4', 'request_id': 'req-7b76eaac-c67d-4fe6-a400-05beda6df99c'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:26.544812 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3743e67-5a25-4dcf-b4d7-a3561863cb39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:26.546995 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3743e67-5a25-4dcf-b4d7-a3561863cb39 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:26.548432 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-7b76eaac-c67d-4fe6-a400-05beda6df99c admin admin] The token cache is being invalidate because project fd760a4c64954833885e2b8d400100f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:27:26.548971 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-7b76eaac-c67d-4fe6-a400-05beda6df99c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:26.549815 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-7b76eaac-c67d-4fe6-a400-05beda6df99c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:26.552520 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1399/2790] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:27:26 2022] DELETE /identity/v3/projects/fd760a4c64954833885e2b8d400100f4 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:27:26.557862 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d514c1-53fa-482b-a642-ca17abda5f07 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:26.558364 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d514c1-53fa-482b-a642-ca17abda5f07 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:26.559421 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d514c1-53fa-482b-a642-ca17abda5f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:26.561604 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1392/2791] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:27:26 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:27:26.581130 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-31d514c1-53fa-482b-a642-ca17abda5f07 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:27:26.582469 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-31d514c1-53fa-482b-a642-ca17abda5f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:26.668127 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1400/2792] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:27:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:27:26.678867 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a619096-b974-4b4c-98f9-920efc2fd094 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:26.679506 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a619096-b974-4b4c-98f9-920efc2fd094 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:26.680377 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a619096-b974-4b4c-98f9-920efc2fd094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:26.691329 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:27:26.716681 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6a619096-b974-4b4c-98f9-920efc2fd094 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:27:26.718162 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6a619096-b974-4b4c-98f9-920efc2fd094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:26.732518 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1393/2793] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:27:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:27:26.736720 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d514c1-53fa-482b-a642-ca17abda5f07 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:26.748219 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d514c1-53fa-482b-a642-ca17abda5f07 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:26.753967 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c972846-c18a-4c2b-99c2-5e85f323cbc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:26.754644 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c972846-c18a-4c2b-99c2-5e85f323cbc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:26.754929 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c972846-c18a-4c2b-99c2-5e85f323cbc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:26.755218 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c972846-c18a-4c2b-99c2-5e85f323cbc1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:26.755959 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c972846-c18a-4c2b-99c2-5e85f323cbc1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:27:26.757153 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c972846-c18a-4c2b-99c2-5e85f323cbc1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:26.758970 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c972846-c18a-4c2b-99c2-5e85f323cbc1 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:26.783732 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1401/2794] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:27:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:27:26.790832 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a619096-b974-4b4c-98f9-920efc2fd094 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:26.813875 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a619096-b974-4b4c-98f9-920efc2fd094 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:26.822431 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b694ffc9-7b53-43d2-beae-566f42d0d048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:26.823221 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b694ffc9-7b53-43d2-beae-566f42d0d048 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:26.823813 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b694ffc9-7b53-43d2-beae-566f42d0d048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:26.824288 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b694ffc9-7b53-43d2-beae-566f42d0d048 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:26.825999 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b694ffc9-7b53-43d2-beae-566f42d0d048 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:26.828234 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b694ffc9-7b53-43d2-beae-566f42d0d048 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:26.869083 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1394/2795] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:27:26 2022] POST /identity/v3/projects => generated 373 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:27:26.870754 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c972846-c18a-4c2b-99c2-5e85f323cbc1 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:26.882815 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c972846-c18a-4c2b-99c2-5e85f323cbc1 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:26.893883 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:27:26.954140 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5c5692-6d20-4345-aac4-167bdbc16654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:26.955029 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c5692-6d20-4345-aac4-167bdbc16654 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:26.955421 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c5692-6d20-4345-aac4-167bdbc16654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:26.955805 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c5692-6d20-4345-aac4-167bdbc16654 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:26.956729 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5c5692-6d20-4345-aac4-167bdbc16654 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:26.958798 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5c5692-6d20-4345-aac4-167bdbc16654 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:26.991255 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1402/2796] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:27:26 2022] POST /identity/v3/users => generated 287 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:27:26.992650 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b694ffc9-7b53-43d2-beae-566f42d0d048 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:27.003908 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b694ffc9-7b53-43d2-beae-566f42d0d048 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:27.009687 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7b7cea-d1ad-4bfa-b31a-38ffe9bfce56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:27.010454 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b7cea-d1ad-4bfa-b31a-38ffe9bfce56 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:27.011898 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b7cea-d1ad-4bfa-b31a-38ffe9bfce56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:27.012268 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b7cea-d1ad-4bfa-b31a-38ffe9bfce56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:27.013107 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7b7cea-d1ad-4bfa-b31a-38ffe9bfce56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:27.014784 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7b7cea-d1ad-4bfa-b31a-38ffe9bfce56 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:27.028496 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1395/2797] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:27:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:27:27.032411 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5c5692-6d20-4345-aac4-167bdbc16654 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:27.048996 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5c5692-6d20-4345-aac4-167bdbc16654 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:27.055918 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd6a529-74af-49e6-84fb-cf1afbb1d0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:27.056750 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd6a529-74af-49e6-84fb-cf1afbb1d0a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:27.057135 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd6a529-74af-49e6-84fb-cf1afbb1d0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:27.057486 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd6a529-74af-49e6-84fb-cf1afbb1d0a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:27.058335 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd6a529-74af-49e6-84fb-cf1afbb1d0a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:27.060069 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd6a529-74af-49e6-84fb-cf1afbb1d0a9 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:27.072881 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1403/2798] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:27:27 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:27:27.076830 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7b7cea-d1ad-4bfa-b31a-38ffe9bfce56 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:27.088445 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7b7cea-d1ad-4bfa-b31a-38ffe9bfce56 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:27.093254 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71672d60-7be7-4289-a973-4dfed4544e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:27.094211 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71672d60-7be7-4289-a973-4dfed4544e26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:27.094815 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71672d60-7be7-4289-a973-4dfed4544e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:27.095165 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71672d60-7be7-4289-a973-4dfed4544e26 admin admin] PATH_INFO: `/v3/projects/850fd289ab144f7e8686162c72c2865d/users/bfc58b7e97f04bc590ff0a9556a0e837/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:27.114048 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71672d60-7be7-4289-a973-4dfed4544e26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=850fd289ab144f7e8686162c72c2865d, user_id=bfc58b7e97f04bc590ff0a9556a0e837, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:27.116903 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71672d60-7be7-4289-a973-4dfed4544e26 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:27.136658 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1396/2799] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:27:27 2022] PUT /identity/v3/projects/850fd289ab144f7e8686162c72c2865d/users/bfc58b7e97f04bc590ff0a9556a0e837/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:27:27.153341 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d4f8aa-19b1-4df2-b4c0-a92dcc05101a None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:27.153680 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d4f8aa-19b1-4df2-b4c0-a92dcc05101a None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:27.154870 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d4f8aa-19b1-4df2-b4c0-a92dcc05101a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:27.195497 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-f7d4f8aa-19b1-4df2-b4c0-a92dcc05101a None None] MFA Rules not processed for user `bfc58b7e97f04bc590ff0a9556a0e837`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:27:27.196808 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f7d4f8aa-19b1-4df2-b4c0-a92dcc05101a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:27.296027 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1404/2800] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:27:27 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:27:27.306819 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71672d60-7be7-4289-a973-4dfed4544e26 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:27.308738 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-71672d60-7be7-4289-a973-4dfed4544e26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:27.414623 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71672d60-7be7-4289-a973-4dfed4544e26 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:27.420183 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5571a4-596a-455b-ab2b-63f69062c88e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:27.420840 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5571a4-596a-455b-ab2b-63f69062c88e service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:27.421179 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5571a4-596a-455b-ab2b-63f69062c88e service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:27.421466 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5571a4-596a-455b-ab2b-63f69062c88e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:27.452366 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5571a4-596a-455b-ab2b-63f69062c88e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:27.454162 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5571a4-596a-455b-ab2b-63f69062c88e service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:27.466558 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1397/2801] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:27:27 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:27:31.789536 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9ce5f-7e35-472d-a10a-54d98c05e63e None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:31.790067 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9ce5f-7e35-472d-a10a-54d98c05e63e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:31.790392 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9ce5f-7e35-472d-a10a-54d98c05e63e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:31.830101 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-99c9ce5f-7e35-472d-a10a-54d98c05e63e None None] MFA Rules not processed for user `7fd3ab8ccfda463f9eed5641141e0c87`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:27:31.831325 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-99c9ce5f-7e35-472d-a10a-54d98c05e63e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:31.918547 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1405/2802] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:27:31 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:27:31.938635 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5571a4-596a-455b-ab2b-63f69062c88e service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:31.940717 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0e5571a4-596a-455b-ab2b-63f69062c88e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:32.021946 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5571a4-596a-455b-ab2b-63f69062c88e service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:32.029527 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a82a817-7f9a-4eba-8e53-a40fa8613f2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15039da209ec445d93d49e2bd6d854eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:32.030384 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82a817-7f9a-4eba-8e53-a40fa8613f2c service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:32.030685 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82a817-7f9a-4eba-8e53-a40fa8613f2c service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:32.032159 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82a817-7f9a-4eba-8e53-a40fa8613f2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:32.034062 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5a82a817-7f9a-4eba-8e53-a40fa8613f2c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:32.079962 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:27:32.102373 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a82a817-7f9a-4eba-8e53-a40fa8613f2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:32.103958 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a82a817-7f9a-4eba-8e53-a40fa8613f2c service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:32.112808 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1398/2803] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:27:31 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:27:33.142042 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a003db9e-0422-4e46-bc4b-b6d190873893 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:33.142562 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a003db9e-0422-4e46-bc4b-b6d190873893 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:33.142702 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a003db9e-0422-4e46-bc4b-b6d190873893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:33.154575 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:27:33.161197 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-a003db9e-0422-4e46-bc4b-b6d190873893 None None] MFA Rules not processed for user `df06c3b8aef249b984bc907b1753747c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:27:33.163512 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a003db9e-0422-4e46-bc4b-b6d190873893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:33.170886 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1406/2804] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:27:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:27:33.368146 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a82a817-7f9a-4eba-8e53-a40fa8613f2c service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:33.370288 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5a82a817-7f9a-4eba-8e53-a40fa8613f2c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:33.503041 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a82a817-7f9a-4eba-8e53-a40fa8613f2c service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:33.510251 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682f5580-3241-48b2-a0de-54b0bbc8e58a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:33.513473 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f5580-3241-48b2-a0de-54b0bbc8e58a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:33.513956 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f5580-3241-48b2-a0de-54b0bbc8e58a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:33.514425 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f5580-3241-48b2-a0de-54b0bbc8e58a admin admin] PATH_INFO: `/v3/users/fcc540ec055a41dca30c17ae29756e18` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:33.515930 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682f5580-3241-48b2-a0de-54b0bbc8e58a admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcc540ec055a41dca30c17ae29756e18)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:33.521622 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682f5580-3241-48b2-a0de-54b0bbc8e58a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:33.610162 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-682f5580-3241-48b2-a0de-54b0bbc8e58a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcc540ec055a41dca30c17ae29756e18'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:33.611494 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-682f5580-3241-48b2-a0de-54b0bbc8e58a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcc540ec055a41dca30c17ae29756e18'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:33.644705 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-682f5580-3241-48b2-a0de-54b0bbc8e58a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcc540ec055a41dca30c17ae29756e18'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:33.667185 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a003db9e-0422-4e46-bc4b-b6d190873893 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:33.670707 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-682f5580-3241-48b2-a0de-54b0bbc8e58a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcc540ec055a41dca30c17ae29756e18'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:33.671676 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-682f5580-3241-48b2-a0de-54b0bbc8e58a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcc540ec055a41dca30c17ae29756e18'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:33.681929 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a003db9e-0422-4e46-bc4b-b6d190873893 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:33.689606 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffab4de-3736-46c2-834e-f64542c7bb20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:33.690691 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffab4de-3736-46c2-834e-f64542c7bb20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:33.691110 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffab4de-3736-46c2-834e-f64542c7bb20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:33.691406 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffab4de-3736-46c2-834e-f64542c7bb20 admin admin] PATH_INFO: `/v3/users/bfc58b7e97f04bc590ff0a9556a0e837` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:33.692875 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffab4de-3736-46c2-834e-f64542c7bb20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfc58b7e97f04bc590ff0a9556a0e837)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:33.695842 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffab4de-3736-46c2-834e-f64542c7bb20 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:33.711086 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1399/2805] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:27:33 2022] DELETE /identity/v3/users/fcc540ec055a41dca30c17ae29756e18 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:27:33.715676 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be950692-2835-4ec7-bc20-b1e5d985db0a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:33.715978 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be950692-2835-4ec7-bc20-b1e5d985db0a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:33.716333 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be950692-2835-4ec7-bc20-b1e5d985db0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:33.759499 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-be950692-2835-4ec7-bc20-b1e5d985db0a None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:27:33.765448 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-be950692-2835-4ec7-bc20-b1e5d985db0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:33.864876 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-2ffab4de-3736-46c2-834e-f64542c7bb20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfc58b7e97f04bc590ff0a9556a0e837'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:33.888341 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:27:33.889108 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-2ffab4de-3736-46c2-834e-f64542c7bb20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfc58b7e97f04bc590ff0a9556a0e837'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:33.889922 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-2ffab4de-3736-46c2-834e-f64542c7bb20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfc58b7e97f04bc590ff0a9556a0e837'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:33.903407 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-2ffab4de-3736-46c2-834e-f64542c7bb20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfc58b7e97f04bc590ff0a9556a0e837'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:33.917750 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1400/2806] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:27:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:27:33.927026 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be950692-2835-4ec7-bc20-b1e5d985db0a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:33.929562 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-be950692-2835-4ec7-bc20-b1e5d985db0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:33.938892 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-2ffab4de-3736-46c2-834e-f64542c7bb20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfc58b7e97f04bc590ff0a9556a0e837'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:33.954150 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1407/2807] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:27:33 2022] DELETE /identity/v3/users/bfc58b7e97f04bc590ff0a9556a0e837 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:27:33.958433 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde81b6e-036a-41a8-aa90-18e2cef8447e None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:33.958821 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde81b6e-036a-41a8-aa90-18e2cef8447e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:33.960964 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde81b6e-036a-41a8-aa90-18e2cef8447e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:33.992314 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-fde81b6e-036a-41a8-aa90-18e2cef8447e None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:27:33.994537 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-fde81b6e-036a-41a8-aa90-18e2cef8447e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:34.042745 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be950692-2835-4ec7-bc20-b1e5d985db0a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:34.097100 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1408/2808] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:27:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:27:34.118230 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde81b6e-036a-41a8-aa90-18e2cef8447e None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:34.122026 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-fde81b6e-036a-41a8-aa90-18e2cef8447e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:34.127110 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5a8840-f314-45f0-90ec-8069ce1e218e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:34.127995 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a8840-f314-45f0-90ec-8069ce1e218e service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:34.128290 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a8840-f314-45f0-90ec-8069ce1e218e service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:34.128617 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a8840-f314-45f0-90ec-8069ce1e218e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:34.155462 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5a8840-f314-45f0-90ec-8069ce1e218e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:34.157606 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5a8840-f314-45f0-90ec-8069ce1e218e service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:34.164926 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde81b6e-036a-41a8-aa90-18e2cef8447e None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:34.198903 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1401/2809] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:27:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:27:34.211989 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c60e79-6f26-4327-adae-5a539ce90c12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:34.212690 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c60e79-6f26-4327-adae-5a539ce90c12 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:34.212985 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c60e79-6f26-4327-adae-5a539ce90c12 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:34.215223 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c60e79-6f26-4327-adae-5a539ce90c12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:34.244560 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c60e79-6f26-4327-adae-5a539ce90c12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:34.250941 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c60e79-6f26-4327-adae-5a539ce90c12 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:34.277121 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1409/2810] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:27:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:27:34.672801 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5a8840-f314-45f0-90ec-8069ce1e218e service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:34.675859 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4f5a8840-f314-45f0-90ec-8069ce1e218e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:34.708915 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5a8840-f314-45f0-90ec-8069ce1e218e service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:34.716483 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d7fa0b-ccbc-482f-a3c0-ae90d9c1e67d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:34.717656 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d7fa0b-ccbc-482f-a3c0-ae90d9c1e67d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:34.718150 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d7fa0b-ccbc-482f-a3c0-ae90d9c1e67d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:34.718910 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d7fa0b-ccbc-482f-a3c0-ae90d9c1e67d admin admin] PATH_INFO: `/v3/projects/33ece32c0c9e48d6b81a8f57d60f214c` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:34.720025 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d7fa0b-ccbc-482f-a3c0-ae90d9c1e67d admin admin] RBAC: Authorizing `identity:delete_project(project_id=33ece32c0c9e48d6b81a8f57d60f214c)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:34.722362 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d7fa0b-ccbc-482f-a3c0-ae90d9c1e67d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:34.737077 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c60e79-6f26-4327-adae-5a539ce90c12 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:34.738833 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-06c60e79-6f26-4327-adae-5a539ce90c12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:34.764477 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c60e79-6f26-4327-adae-5a539ce90c12 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:34.777793 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d3fc35-5d7e-4c46-9977-e0d81e8e2ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:34.778754 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d3fc35-5d7e-4c46-9977-e0d81e8e2ae9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:34.779050 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d3fc35-5d7e-4c46-9977-e0d81e8e2ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:34.779377 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d3fc35-5d7e-4c46-9977-e0d81e8e2ae9 admin admin] PATH_INFO: `/v3/projects/850fd289ab144f7e8686162c72c2865d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:34.782794 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d3fc35-5d7e-4c46-9977-e0d81e8e2ae9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=850fd289ab144f7e8686162c72c2865d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:34.785348 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d3fc35-5d7e-4c46-9977-e0d81e8e2ae9 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:34.858819 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f8d7fa0b-ccbc-482f-a3c0-ae90d9c1e67d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33ece32c0c9e48d6b81a8f57d60f214c', 'request_id': 'req-f8d7fa0b-ccbc-482f-a3c0-ae90d9c1e67d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:34.870082 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f8d7fa0b-ccbc-482f-a3c0-ae90d9c1e67d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33ece32c0c9e48d6b81a8f57d60f214c', 'request_id': 'req-f8d7fa0b-ccbc-482f-a3c0-ae90d9c1e67d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:34.889950 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f8d7fa0b-ccbc-482f-a3c0-ae90d9c1e67d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33ece32c0c9e48d6b81a8f57d60f214c', 'request_id': 'req-f8d7fa0b-ccbc-482f-a3c0-ae90d9c1e67d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:34.930143 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f8d7fa0b-ccbc-482f-a3c0-ae90d9c1e67d admin admin] The token cache is being invalidate because project 33ece32c0c9e48d6b81a8f57d60f214c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:27:34.930781 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f8d7fa0b-ccbc-482f-a3c0-ae90d9c1e67d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:34.932035 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f8d7fa0b-ccbc-482f-a3c0-ae90d9c1e67d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:34.939932 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1402/2811] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:27:34 2022] DELETE /identity/v3/projects/33ece32c0c9e48d6b81a8f57d60f214c => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:27:34.946168 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d7fa0b-ccbc-482f-a3c0-ae90d9c1e67d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:34.947851 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f8d7fa0b-ccbc-482f-a3c0-ae90d9c1e67d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:34.949381 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-78d3fc35-5d7e-4c46-9977-e0d81e8e2ae9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '850fd289ab144f7e8686162c72c2865d', 'request_id': 'req-78d3fc35-5d7e-4c46-9977-e0d81e8e2ae9'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:34.963863 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:27:34.974727 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-78d3fc35-5d7e-4c46-9977-e0d81e8e2ae9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '850fd289ab144f7e8686162c72c2865d', 'request_id': 'req-78d3fc35-5d7e-4c46-9977-e0d81e8e2ae9'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:34.989709 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-78d3fc35-5d7e-4c46-9977-e0d81e8e2ae9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '850fd289ab144f7e8686162c72c2865d', 'request_id': 'req-78d3fc35-5d7e-4c46-9977-e0d81e8e2ae9'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:35.030283 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-78d3fc35-5d7e-4c46-9977-e0d81e8e2ae9 admin admin] The token cache is being invalidate because project 850fd289ab144f7e8686162c72c2865d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:27:35.031191 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-78d3fc35-5d7e-4c46-9977-e0d81e8e2ae9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:35.044828 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-78d3fc35-5d7e-4c46-9977-e0d81e8e2ae9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:35.047051 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1410/2812] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:27:34 2022] DELETE /identity/v3/projects/850fd289ab144f7e8686162c72c2865d => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:27:35.048587 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:27:35.054037 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e7befa-8cdd-46ff-92d2-160c773cace6 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:35.054457 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e7befa-8cdd-46ff-92d2-160c773cace6 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:35.054710 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e7befa-8cdd-46ff-92d2-160c773cace6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:35.081626 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d7fa0b-ccbc-482f-a3c0-ae90d9c1e67d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:35.082104 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-33e7befa-8cdd-46ff-92d2-160c773cace6 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:27:35.083350 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-33e7befa-8cdd-46ff-92d2-160c773cace6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:35.149607 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3949a8-df76-4998-a090-3cf422eb431f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:35.150694 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3949a8-df76-4998-a090-3cf422eb431f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:35.150977 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3949a8-df76-4998-a090-3cf422eb431f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:35.151254 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3949a8-df76-4998-a090-3cf422eb431f admin admin] PATH_INFO: `/v3/users/2999125ffbd64ef79ff6e97045d859e6` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:35.152524 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3949a8-df76-4998-a090-3cf422eb431f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2999125ffbd64ef79ff6e97045d859e6)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:35.154508 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3949a8-df76-4998-a090-3cf422eb431f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:35.181100 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1411/2813] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:27:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:27:35.190148 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c4ad7-c8d6-4ad6-b926-f70398455a96 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:35.190454 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c4ad7-c8d6-4ad6-b926-f70398455a96 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:35.190821 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c4ad7-c8d6-4ad6-b926-f70398455a96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:35.216679 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-461c4ad7-c8d6-4ad6-b926-f70398455a96 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:27:35.218540 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-461c4ad7-c8d6-4ad6-b926-f70398455a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:35.222204 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cb3949a8-df76-4998-a090-3cf422eb431f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2999125ffbd64ef79ff6e97045d859e6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:35.222945 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cb3949a8-df76-4998-a090-3cf422eb431f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2999125ffbd64ef79ff6e97045d859e6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:35.233538 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cb3949a8-df76-4998-a090-3cf422eb431f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2999125ffbd64ef79ff6e97045d859e6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:35.233767 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1412/2814] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:27:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:27:35.237766 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c4ad7-c8d6-4ad6-b926-f70398455a96 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:35.247719 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cb3949a8-df76-4998-a090-3cf422eb431f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2999125ffbd64ef79ff6e97045d859e6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:35.248717 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cb3949a8-df76-4998-a090-3cf422eb431f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2999125ffbd64ef79ff6e97045d859e6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:35.248844 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c4ad7-c8d6-4ad6-b926-f70398455a96 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:35.255542 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c5d175-c98e-4a25-8755-2ae6c92efe66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:35.257219 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c5d175-c98e-4a25-8755-2ae6c92efe66 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:35.258125 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c5d175-c98e-4a25-8755-2ae6c92efe66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:35.258355 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c5d175-c98e-4a25-8755-2ae6c92efe66 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:35.260750 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c5d175-c98e-4a25-8755-2ae6c92efe66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:27:35.261098 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c5d175-c98e-4a25-8755-2ae6c92efe66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:35.262959 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c5d175-c98e-4a25-8755-2ae6c92efe66 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:35.266422 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1403/2815] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:27:34 2022] DELETE /identity/v3/users/2999125ffbd64ef79ff6e97045d859e6 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:27:35.281450 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1413/2816] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:27:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:27:35.283699 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3949a8-df76-4998-a090-3cf422eb431f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:35.289269 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cb3949a8-df76-4998-a090-3cf422eb431f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:35.414411 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3949a8-df76-4998-a090-3cf422eb431f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:35.432059 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33895d27-fc4d-4c9d-b39a-9ed3984f674f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:35.436647 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33895d27-fc4d-4c9d-b39a-9ed3984f674f admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:35.436647 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33895d27-fc4d-4c9d-b39a-9ed3984f674f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:35.436647 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33895d27-fc4d-4c9d-b39a-9ed3984f674f admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:35.436647 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33895d27-fc4d-4c9d-b39a-9ed3984f674f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:35.436647 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33895d27-fc4d-4c9d-b39a-9ed3984f674f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:35.458003 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:27:35.495143 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1404/2817] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:27:35 2022] POST /identity/v3/projects => generated 379 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:27:35.499604 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c5d175-c98e-4a25-8755-2ae6c92efe66 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:35.511015 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c5d175-c98e-4a25-8755-2ae6c92efe66 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:35.610667 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae84efa-1153-43ef-9abb-ef41b0f18d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:35.611360 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae84efa-1153-43ef-9abb-ef41b0f18d97 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:35.611647 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae84efa-1153-43ef-9abb-ef41b0f18d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:35.611928 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae84efa-1153-43ef-9abb-ef41b0f18d97 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:35.612729 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae84efa-1153-43ef-9abb-ef41b0f18d97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:35.614787 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae84efa-1153-43ef-9abb-ef41b0f18d97 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:35.651829 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1414/2818] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:27:35 2022] POST /identity/v3/users => generated 290 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:27:35.655758 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33895d27-fc4d-4c9d-b39a-9ed3984f674f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:35.674318 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33895d27-fc4d-4c9d-b39a-9ed3984f674f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:35.680838 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7203ed-bb1f-412a-9dc8-cf72262ea917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:35.681999 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7203ed-bb1f-412a-9dc8-cf72262ea917 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:35.682443 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7203ed-bb1f-412a-9dc8-cf72262ea917 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:35.682772 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7203ed-bb1f-412a-9dc8-cf72262ea917 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:35.683783 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7203ed-bb1f-412a-9dc8-cf72262ea917 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:35.685607 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7203ed-bb1f-412a-9dc8-cf72262ea917 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:35.690185 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae84efa-1153-43ef-9abb-ef41b0f18d97 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:35.692099 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-cae84efa-1153-43ef-9abb-ef41b0f18d97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:35.708487 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1405/2819] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:27:35 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:27:35.711411 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae84efa-1153-43ef-9abb-ef41b0f18d97 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:35.714289 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7203ed-bb1f-412a-9dc8-cf72262ea917 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:35.722371 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4032935d-8877-4738-a266-97c71793d589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:35.723548 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032935d-8877-4738-a266-97c71793d589 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:35.724090 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032935d-8877-4738-a266-97c71793d589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:35.724383 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032935d-8877-4738-a266-97c71793d589 admin admin] PATH_INFO: `/v3/projects/fdba18a4556547b39b35723ae54a1ef0` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:35.726128 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4032935d-8877-4738-a266-97c71793d589 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdba18a4556547b39b35723ae54a1ef0)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:35.728203 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4032935d-8877-4738-a266-97c71793d589 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:35.729593 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7203ed-bb1f-412a-9dc8-cf72262ea917 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:35.736550 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d92bdc-eaa4-49d2-a9aa-3a1ba41380b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:35.737992 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d92bdc-eaa4-49d2-a9aa-3a1ba41380b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:35.738563 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d92bdc-eaa4-49d2-a9aa-3a1ba41380b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:35.739051 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d92bdc-eaa4-49d2-a9aa-3a1ba41380b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:35.740015 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d92bdc-eaa4-49d2-a9aa-3a1ba41380b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:35.742303 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d92bdc-eaa4-49d2-a9aa-3a1ba41380b0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:35.756557 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1406/2820] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:27:35 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:27:35.761178 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d92bdc-eaa4-49d2-a9aa-3a1ba41380b0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:35.773465 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d92bdc-eaa4-49d2-a9aa-3a1ba41380b0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:35.792461 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:27:35.795231 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4032935d-8877-4738-a266-97c71793d589 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdba18a4556547b39b35723ae54a1ef0', 'request_id': 'req-4032935d-8877-4738-a266-97c71793d589'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:35.804828 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4032935d-8877-4738-a266-97c71793d589 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdba18a4556547b39b35723ae54a1ef0', 'request_id': 'req-4032935d-8877-4738-a266-97c71793d589'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:35.813710 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4032935d-8877-4738-a266-97c71793d589 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdba18a4556547b39b35723ae54a1ef0', 'request_id': 'req-4032935d-8877-4738-a266-97c71793d589'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:35.831815 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4032935d-8877-4738-a266-97c71793d589 admin admin] The token cache is being invalidate because project fdba18a4556547b39b35723ae54a1ef0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:27:35.832130 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4032935d-8877-4738-a266-97c71793d589 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:35.832964 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4032935d-8877-4738-a266-97c71793d589 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:35.836394 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1415/2821] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:27:35 2022] DELETE /identity/v3/projects/fdba18a4556547b39b35723ae54a1ef0 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:27:35.842165 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4032935d-8877-4738-a266-97c71793d589 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:35.845613 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4032935d-8877-4738-a266-97c71793d589 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:35.863297 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf3c1f6-c409-48f6-95f9-8dc91501d3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:35.864211 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf3c1f6-c409-48f6-95f9-8dc91501d3e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:35.864505 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf3c1f6-c409-48f6-95f9-8dc91501d3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:35.864786 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf3c1f6-c409-48f6-95f9-8dc91501d3e6 admin admin] PATH_INFO: `/v3/projects/4fe236d825ae40f793bf2a5bccedc3cd/users/70733d7f6b8b42a39425ebc94e1ec1f4/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:35.892740 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf3c1f6-c409-48f6-95f9-8dc91501d3e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fe236d825ae40f793bf2a5bccedc3cd, user_id=70733d7f6b8b42a39425ebc94e1ec1f4, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:35.894015 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf3c1f6-c409-48f6-95f9-8dc91501d3e6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:35.916410 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1407/2822] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:27:35 2022] PUT /identity/v3/projects/4fe236d825ae40f793bf2a5bccedc3cd/users/70733d7f6b8b42a39425ebc94e1ec1f4/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:27:35.921960 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec8a452-fcd6-48cb-9268-6a403ee9e4de None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:35.922256 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec8a452-fcd6-48cb-9268-6a403ee9e4de None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:35.922540 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec8a452-fcd6-48cb-9268-6a403ee9e4de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:35.935796 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4032935d-8877-4738-a266-97c71793d589 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:35.945161 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-cec8a452-fcd6-48cb-9268-6a403ee9e4de None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:27:35.946620 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cec8a452-fcd6-48cb-9268-6a403ee9e4de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:36.010429 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d36613-ef06-4e15-82ad-45c1454427b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:36.011397 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d36613-ef06-4e15-82ad-45c1454427b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:36.012836 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d36613-ef06-4e15-82ad-45c1454427b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:36.013175 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d36613-ef06-4e15-82ad-45c1454427b6 admin admin] PATH_INFO: `/v3/users/1d32d8f4131b4df2ad68b413115fbfc2` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:36.014491 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d36613-ef06-4e15-82ad-45c1454427b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d32d8f4131b4df2ad68b413115fbfc2)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:36.016497 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d36613-ef06-4e15-82ad-45c1454427b6 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:36.027737 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1408/2823] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:27:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:27:36.038501 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec8a452-fcd6-48cb-9268-6a403ee9e4de None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:36.040192 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cec8a452-fcd6-48cb-9268-6a403ee9e4de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:36.076855 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c1d36613-ef06-4e15-82ad-45c1454427b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d32d8f4131b4df2ad68b413115fbfc2'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:36.091827 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c1d36613-ef06-4e15-82ad-45c1454427b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d32d8f4131b4df2ad68b413115fbfc2'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:36.092668 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c1d36613-ef06-4e15-82ad-45c1454427b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d32d8f4131b4df2ad68b413115fbfc2'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:36.102077 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c1d36613-ef06-4e15-82ad-45c1454427b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d32d8f4131b4df2ad68b413115fbfc2'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:36.124327 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c1d36613-ef06-4e15-82ad-45c1454427b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d32d8f4131b4df2ad68b413115fbfc2'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:36.127273 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec8a452-fcd6-48cb-9268-6a403ee9e4de None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:36.135002 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1416/2824] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:27:35 2022] DELETE /identity/v3/users/1d32d8f4131b4df2ad68b413115fbfc2 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:27:36.137077 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7668a6df-0c68-4afb-9b95-92a5f9840b4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:36.141653 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7668a6df-0c68-4afb-9b95-92a5f9840b4a service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:36.142274 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7668a6df-0c68-4afb-9b95-92a5f9840b4a service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:36.142904 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7668a6df-0c68-4afb-9b95-92a5f9840b4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:36.145137 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7668a6df-0c68-4afb-9b95-92a5f9840b4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:36.156574 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:27:36.159759 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7668a6df-0c68-4afb-9b95-92a5f9840b4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:36.161638 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7668a6df-0c68-4afb-9b95-92a5f9840b4a service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:36.180471 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1409/2825] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:27:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:27:36.516940 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d36613-ef06-4e15-82ad-45c1454427b6 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:36.518889 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c1d36613-ef06-4e15-82ad-45c1454427b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:36.525617 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:27:36.643157 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d36613-ef06-4e15-82ad-45c1454427b6 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:36.649819 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a72f777-3125-4a40-b7b7-258797d7c15d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:36.651263 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a72f777-3125-4a40-b7b7-258797d7c15d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:36.651263 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a72f777-3125-4a40-b7b7-258797d7c15d admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:36.651508 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a72f777-3125-4a40-b7b7-258797d7c15d admin admin] PATH_INFO: `/v3/projects/5ff88fefbf5c41f384872f427a9561d9` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:36.653830 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a72f777-3125-4a40-b7b7-258797d7c15d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ff88fefbf5c41f384872f427a9561d9)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:36.661831 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a72f777-3125-4a40-b7b7-258797d7c15d admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:36.731196 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-1a72f777-3125-4a40-b7b7-258797d7c15d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ff88fefbf5c41f384872f427a9561d9', 'request_id': 'req-1a72f777-3125-4a40-b7b7-258797d7c15d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:36.741529 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-1a72f777-3125-4a40-b7b7-258797d7c15d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ff88fefbf5c41f384872f427a9561d9', 'request_id': 'req-1a72f777-3125-4a40-b7b7-258797d7c15d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:36.748665 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-1a72f777-3125-4a40-b7b7-258797d7c15d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ff88fefbf5c41f384872f427a9561d9', 'request_id': 'req-1a72f777-3125-4a40-b7b7-258797d7c15d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:36.767683 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-1a72f777-3125-4a40-b7b7-258797d7c15d admin admin] The token cache is being invalidate because project 5ff88fefbf5c41f384872f427a9561d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:27:36.767950 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-1a72f777-3125-4a40-b7b7-258797d7c15d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:36.769638 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-1a72f777-3125-4a40-b7b7-258797d7c15d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:36.772317 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1417/2826] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:27:36 2022] DELETE /identity/v3/projects/5ff88fefbf5c41f384872f427a9561d9 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:27:36.777821 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db912873-bc78-42b3-85c9-f7784ef75e1c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:36.778347 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db912873-bc78-42b3-85c9-f7784ef75e1c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:36.778504 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db912873-bc78-42b3-85c9-f7784ef75e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:36.800793 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-db912873-bc78-42b3-85c9-f7784ef75e1c None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:27:36.802022 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-db912873-bc78-42b3-85c9-f7784ef75e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:36.876460 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1410/2827] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:27:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:27:36.884894 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e562bd2-0c5c-461a-9033-341cfeaa23cf None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:36.886578 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e562bd2-0c5c-461a-9033-341cfeaa23cf None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:36.886758 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e562bd2-0c5c-461a-9033-341cfeaa23cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:36.917936 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-6e562bd2-0c5c-461a-9033-341cfeaa23cf None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:27:36.919407 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-6e562bd2-0c5c-461a-9033-341cfeaa23cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:36.930664 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1418/2828] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:27:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:27:36.935894 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db912873-bc78-42b3-85c9-f7784ef75e1c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:36.951569 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db912873-bc78-42b3-85c9-f7784ef75e1c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:36.965350 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0621900-110c-45ee-a506-33f619077406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:36.966465 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0621900-110c-45ee-a506-33f619077406 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:36.966859 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0621900-110c-45ee-a506-33f619077406 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:36.967322 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0621900-110c-45ee-a506-33f619077406 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:36.968721 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0621900-110c-45ee-a506-33f619077406 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:27:36.969204 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0621900-110c-45ee-a506-33f619077406 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:36.972682 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0621900-110c-45ee-a506-33f619077406 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:36.997162 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1411/2829] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:27:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:27:36.997967 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e562bd2-0c5c-461a-9033-341cfeaa23cf None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:37.016274 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e562bd2-0c5c-461a-9033-341cfeaa23cf None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:37.022244 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f56734-e7fd-428c-8082-25ec86c7346f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:37.023029 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f56734-e7fd-428c-8082-25ec86c7346f admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:37.023497 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f56734-e7fd-428c-8082-25ec86c7346f admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:37.023906 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f56734-e7fd-428c-8082-25ec86c7346f admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:37.024491 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f56734-e7fd-428c-8082-25ec86c7346f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:37.026494 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f56734-e7fd-428c-8082-25ec86c7346f admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:37.069550 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1419/2830] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:27:36 2022] POST /identity/v3/projects => generated 385 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:27:37.073058 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0621900-110c-45ee-a506-33f619077406 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:37.085453 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0621900-110c-45ee-a506-33f619077406 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:37.170732 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22770ce2-9362-4b9c-ade0-92a03c1022f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:37.172330 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22770ce2-9362-4b9c-ade0-92a03c1022f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:37.172744 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22770ce2-9362-4b9c-ade0-92a03c1022f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:37.173131 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22770ce2-9362-4b9c-ade0-92a03c1022f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:37.174094 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22770ce2-9362-4b9c-ade0-92a03c1022f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:37.180343 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22770ce2-9362-4b9c-ade0-92a03c1022f9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:37.215922 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1412/2831] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:27:37 2022] POST /identity/v3/users => generated 293 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:27:37.217442 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f56734-e7fd-428c-8082-25ec86c7346f admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:37.230576 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f56734-e7fd-428c-8082-25ec86c7346f admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:37.235489 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a371a01e-49a5-431b-acda-6459953048ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:37.236227 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371a01e-49a5-431b-acda-6459953048ff admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:37.236600 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371a01e-49a5-431b-acda-6459953048ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:37.236942 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371a01e-49a5-431b-acda-6459953048ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:37.237779 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a371a01e-49a5-431b-acda-6459953048ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:37.239482 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a371a01e-49a5-431b-acda-6459953048ff admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:37.254225 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1420/2832] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:27:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:27:37.255493 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22770ce2-9362-4b9c-ade0-92a03c1022f9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:37.266940 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22770ce2-9362-4b9c-ade0-92a03c1022f9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:37.273071 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454b6c25-4fa3-42ae-aaba-3e72f89c8335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:37.274067 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454b6c25-4fa3-42ae-aaba-3e72f89c8335 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:37.274524 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454b6c25-4fa3-42ae-aaba-3e72f89c8335 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:37.274889 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454b6c25-4fa3-42ae-aaba-3e72f89c8335 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:37.275912 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454b6c25-4fa3-42ae-aaba-3e72f89c8335 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:37.278139 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454b6c25-4fa3-42ae-aaba-3e72f89c8335 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:37.300347 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1413/2833] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:27:37 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:27:37.302270 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a371a01e-49a5-431b-acda-6459953048ff admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:37.318158 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a371a01e-49a5-431b-acda-6459953048ff admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:37.325194 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c708c1-6c3a-4012-bf89-8c1fea785807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:37.326045 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c708c1-6c3a-4012-bf89-8c1fea785807 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:37.326334 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c708c1-6c3a-4012-bf89-8c1fea785807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:37.326617 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c708c1-6c3a-4012-bf89-8c1fea785807 admin admin] PATH_INFO: `/v3/projects/75a4ae29fb0040c3b510bb52127181fb/users/57e732c6290a460da4345e5b30cefd04/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:37.344917 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c708c1-6c3a-4012-bf89-8c1fea785807 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75a4ae29fb0040c3b510bb52127181fb, user_id=57e732c6290a460da4345e5b30cefd04, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:37.347400 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c708c1-6c3a-4012-bf89-8c1fea785807 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:37.370833 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1421/2834] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:27:37 2022] PUT /identity/v3/projects/75a4ae29fb0040c3b510bb52127181fb/users/57e732c6290a460da4345e5b30cefd04/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:27:37.382285 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be0aae9-16b0-47af-8a43-a648246c10df None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:37.382591 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be0aae9-16b0-47af-8a43-a648246c10df None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:37.382861 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be0aae9-16b0-47af-8a43-a648246c10df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:37.418641 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5be0aae9-16b0-47af-8a43-a648246c10df None None] MFA Rules not processed for user `57e732c6290a460da4345e5b30cefd04`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:27:37.420008 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5be0aae9-16b0-47af-8a43-a648246c10df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:37.478214 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:27:37.505213 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1414/2835] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:27:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:27:37.517379 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c708c1-6c3a-4012-bf89-8c1fea785807 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:37.519459 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b7c708c1-6c3a-4012-bf89-8c1fea785807 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:37.573411 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:27:37.619173 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c708c1-6c3a-4012-bf89-8c1fea785807 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:37.625781 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cbb9af-9e22-44c5-846f-a3805c466006 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:37.626790 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cbb9af-9e22-44c5-846f-a3805c466006 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:37.626790 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cbb9af-9e22-44c5-846f-a3805c466006 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:37.627020 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cbb9af-9e22-44c5-846f-a3805c466006 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:37.637510 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cbb9af-9e22-44c5-846f-a3805c466006 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:37.638952 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cbb9af-9e22-44c5-846f-a3805c466006 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:37.650177 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1422/2836] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:27:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:27:40.610552 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be0aae9-16b0-47af-8a43-a648246c10df None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:40.676973 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be0aae9-16b0-47af-8a43-a648246c10df None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:40.683027 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f166c4-ae7e-4bbe-883b-88035ebd1dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:40.683987 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f166c4-ae7e-4bbe-883b-88035ebd1dd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:40.684367 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f166c4-ae7e-4bbe-883b-88035ebd1dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:40.684657 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f166c4-ae7e-4bbe-883b-88035ebd1dd8 admin admin] PATH_INFO: `/v3/users/57e732c6290a460da4345e5b30cefd04` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:40.686338 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f166c4-ae7e-4bbe-883b-88035ebd1dd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57e732c6290a460da4345e5b30cefd04)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:40.688381 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f166c4-ae7e-4bbe-883b-88035ebd1dd8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:40.742194 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b0f166c4-ae7e-4bbe-883b-88035ebd1dd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57e732c6290a460da4345e5b30cefd04'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:40.742852 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b0f166c4-ae7e-4bbe-883b-88035ebd1dd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57e732c6290a460da4345e5b30cefd04'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:40.763075 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b0f166c4-ae7e-4bbe-883b-88035ebd1dd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57e732c6290a460da4345e5b30cefd04'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:40.785654 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b0f166c4-ae7e-4bbe-883b-88035ebd1dd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57e732c6290a460da4345e5b30cefd04'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:40.786496 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b0f166c4-ae7e-4bbe-883b-88035ebd1dd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57e732c6290a460da4345e5b30cefd04'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:40.800054 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1415/2837] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:27:40 2022] DELETE /identity/v3/users/57e732c6290a460da4345e5b30cefd04 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:27:40.805339 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460512ea-4c7f-433d-b122-1a772ba3137c None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:40.805824 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460512ea-4c7f-433d-b122-1a772ba3137c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:40.805926 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460512ea-4c7f-433d-b122-1a772ba3137c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:40.825464 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-460512ea-4c7f-433d-b122-1a772ba3137c None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:27:40.826695 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-460512ea-4c7f-433d-b122-1a772ba3137c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:40.904302 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1423/2838] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:27:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:27:40.913566 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f166c4-ae7e-4bbe-883b-88035ebd1dd8 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:40.915344 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b0f166c4-ae7e-4bbe-883b-88035ebd1dd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:40.990726 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f166c4-ae7e-4bbe-883b-88035ebd1dd8 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:40.995272 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e3b361-318f-4b75-abae-0a08b74b6335 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:40.997101 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e3b361-318f-4b75-abae-0a08b74b6335 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:40.997562 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e3b361-318f-4b75-abae-0a08b74b6335 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:40.998118 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e3b361-318f-4b75-abae-0a08b74b6335 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:41.011813 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e3b361-318f-4b75-abae-0a08b74b6335 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:41.013387 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e3b361-318f-4b75-abae-0a08b74b6335 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:41.025192 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1416/2839] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:27:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:27:41.115534 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425661c2-6cdd-4f9a-9953-3f0e079f566b None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:41.115792 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425661c2-6cdd-4f9a-9953-3f0e079f566b None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:41.115905 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425661c2-6cdd-4f9a-9953-3f0e079f566b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:41.146260 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-425661c2-6cdd-4f9a-9953-3f0e079f566b None None] MFA Rules not processed for user `70733d7f6b8b42a39425ebc94e1ec1f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:27:41.148186 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-425661c2-6cdd-4f9a-9953-3f0e079f566b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:41.258705 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1424/2840] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:27:41 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:27:41.263650 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e3b361-318f-4b75-abae-0a08b74b6335 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:41.267727 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-72e3b361-318f-4b75-abae-0a08b74b6335 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:41.306342 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e3b361-318f-4b75-abae-0a08b74b6335 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:41.315953 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fd7a88-37e9-483f-957e-8b0932fab4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:41.316645 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd7a88-37e9-483f-957e-8b0932fab4c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:41.317140 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd7a88-37e9-483f-957e-8b0932fab4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:41.317573 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd7a88-37e9-483f-957e-8b0932fab4c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:41.319009 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fd7a88-37e9-483f-957e-8b0932fab4c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:41.321009 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fd7a88-37e9-483f-957e-8b0932fab4c5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:41.332261 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:27:41.353023 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1417/2841] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:27:41 2022] POST /identity/v3/projects => generated 379 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:27:41.356871 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425661c2-6cdd-4f9a-9953-3f0e079f566b None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:41.374020 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425661c2-6cdd-4f9a-9953-3f0e079f566b None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:41.456928 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fd7a88-37e9-483f-957e-8b0932fab4c5 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:41.463929 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-07fd7a88-37e9-483f-957e-8b0932fab4c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:41.502155 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ba65fb-5c51-45f9-8864-5dc38b7bad2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:41.502897 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba65fb-5c51-45f9-8864-5dc38b7bad2e admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:41.503185 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba65fb-5c51-45f9-8864-5dc38b7bad2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:41.503464 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba65fb-5c51-45f9-8864-5dc38b7bad2e admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:41.504174 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ba65fb-5c51-45f9-8864-5dc38b7bad2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:41.510215 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ba65fb-5c51-45f9-8864-5dc38b7bad2e admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:41.522571 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:27:41.553095 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1425/2842] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:27:41 2022] POST /identity/v3/users => generated 296 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:27:41.554918 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ba65fb-5c51-45f9-8864-5dc38b7bad2e admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:41.569419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ba65fb-5c51-45f9-8864-5dc38b7bad2e admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:41.578164 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b9a944-776a-40ab-8f36-16b5c031b632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:41.578853 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b9a944-776a-40ab-8f36-16b5c031b632 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:41.579145 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b9a944-776a-40ab-8f36-16b5c031b632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:41.579430 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b9a944-776a-40ab-8f36-16b5c031b632 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:41.580174 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b9a944-776a-40ab-8f36-16b5c031b632 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:41.582211 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b9a944-776a-40ab-8f36-16b5c031b632 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:41.587868 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fd7a88-37e9-483f-957e-8b0932fab4c5 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:41.596406 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802a2d3d-4994-4dbd-a69d-156c9083ae32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:41.597418 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a2d3d-4994-4dbd-a69d-156c9083ae32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:41.597858 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a2d3d-4994-4dbd-a69d-156c9083ae32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:41.598141 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a2d3d-4994-4dbd-a69d-156c9083ae32 admin admin] PATH_INFO: `/v3/projects/75a4ae29fb0040c3b510bb52127181fb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:41.602444 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1426/2843] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:27:41 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:27:41.605823 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802a2d3d-4994-4dbd-a69d-156c9083ae32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75a4ae29fb0040c3b510bb52127181fb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:41.607638 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b9a944-776a-40ab-8f36-16b5c031b632 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:41.607826 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802a2d3d-4994-4dbd-a69d-156c9083ae32 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:41.638471 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b9a944-776a-40ab-8f36-16b5c031b632 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:41.646030 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f62db1b-c179-4545-9d33-16d9a30d73fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:41.647257 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f62db1b-c179-4545-9d33-16d9a30d73fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:41.647890 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f62db1b-c179-4545-9d33-16d9a30d73fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:41.648274 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f62db1b-c179-4545-9d33-16d9a30d73fe admin admin] PATH_INFO: `/v3/projects/cc1b69d242fd470690760c789911c41a/users/45747dabd8254bbea4dddd514647c4e4/roles/f36e053b442540f08041ac7878760f65` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:41.675214 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f62db1b-c179-4545-9d33-16d9a30d73fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc1b69d242fd470690760c789911c41a, user_id=45747dabd8254bbea4dddd514647c4e4, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:41.681177 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f62db1b-c179-4545-9d33-16d9a30d73fe admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:41.684045 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-802a2d3d-4994-4dbd-a69d-156c9083ae32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75a4ae29fb0040c3b510bb52127181fb', 'request_id': 'req-802a2d3d-4994-4dbd-a69d-156c9083ae32'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:41.691219 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-802a2d3d-4994-4dbd-a69d-156c9083ae32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75a4ae29fb0040c3b510bb52127181fb', 'request_id': 'req-802a2d3d-4994-4dbd-a69d-156c9083ae32'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:41.701947 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-802a2d3d-4994-4dbd-a69d-156c9083ae32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75a4ae29fb0040c3b510bb52127181fb', 'request_id': 'req-802a2d3d-4994-4dbd-a69d-156c9083ae32'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:41.706325 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1427/2844] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:27:41 2022] PUT /identity/v3/projects/cc1b69d242fd470690760c789911c41a/users/45747dabd8254bbea4dddd514647c4e4/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:27:41.710299 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f62db1b-c179-4545-9d33-16d9a30d73fe admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:41.729445 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-802a2d3d-4994-4dbd-a69d-156c9083ae32 admin admin] The token cache is being invalidate because project 75a4ae29fb0040c3b510bb52127181fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:27:41.729675 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-802a2d3d-4994-4dbd-a69d-156c9083ae32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:41.730683 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-802a2d3d-4994-4dbd-a69d-156c9083ae32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:27:41.732712 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1418/2845] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:27:41 2022] DELETE /identity/v3/projects/75a4ae29fb0040c3b510bb52127181fb => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:27:41.737719 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda0a5cc-f098-4b10-8beb-3609c8e6d465 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:41.738057 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda0a5cc-f098-4b10-8beb-3609c8e6d465 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:41.738229 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda0a5cc-f098-4b10-8beb-3609c8e6d465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:41.767307 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-fda0a5cc-f098-4b10-8beb-3609c8e6d465 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:27:41.768362 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fda0a5cc-f098-4b10-8beb-3609c8e6d465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:41.806910 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f62db1b-c179-4545-9d33-16d9a30d73fe admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:41.815690 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e2a33e-86fa-483b-b035-f1a73136d108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:41.822277 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2a33e-86fa-483b-b035-f1a73136d108 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:41.822499 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2a33e-86fa-483b-b035-f1a73136d108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:41.822710 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2a33e-86fa-483b-b035-f1a73136d108 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:41.823476 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e2a33e-86fa-483b-b035-f1a73136d108 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:41.825385 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e2a33e-86fa-483b-b035-f1a73136d108 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:41.839049 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1428/2846] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:27:41 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:27:41.845500 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e2a33e-86fa-483b-b035-f1a73136d108 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:41.847438 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-20e2a33e-86fa-483b-b035-f1a73136d108 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:41.860594 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1419/2847] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:27:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:27:41.869947 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a2dc7-c53e-4312-8afb-6fe1bded41d8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:41.870238 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a2dc7-c53e-4312-8afb-6fe1bded41d8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:41.870553 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a2dc7-c53e-4312-8afb-6fe1bded41d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:41.871708 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e2a33e-86fa-483b-b035-f1a73136d108 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:41.876965 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85534a2e-61d6-4f3f-acb5-d68a29ae1363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:41.879161 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85534a2e-61d6-4f3f-acb5-d68a29ae1363 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:41.879161 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85534a2e-61d6-4f3f-acb5-d68a29ae1363 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:41.879161 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85534a2e-61d6-4f3f-acb5-d68a29ae1363 admin admin] PATH_INFO: `/v3/projects/cc1b69d242fd470690760c789911c41a/users/45747dabd8254bbea4dddd514647c4e4/roles/f36e053b442540f08041ac7878760f65` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:41.881310 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85534a2e-61d6-4f3f-acb5-d68a29ae1363 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc1b69d242fd470690760c789911c41a, user_id=45747dabd8254bbea4dddd514647c4e4, role_id=f36e053b442540f08041ac7878760f65)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:41.884011 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85534a2e-61d6-4f3f-acb5-d68a29ae1363 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:41.893656 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0c7a2dc7-c53e-4312-8afb-6fe1bded41d8 None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:27:41.895008 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0c7a2dc7-c53e-4312-8afb-6fe1bded41d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:41.900580 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1429/2848] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:27:41 2022] PUT /identity/v3/projects/cc1b69d242fd470690760c789911c41a/users/45747dabd8254bbea4dddd514647c4e4/roles/f36e053b442540f08041ac7878760f65 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:27:41.925848 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:27:41.992375 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1420/2849] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:27:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:27:41.996810 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85534a2e-61d6-4f3f-acb5-d68a29ae1363 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:42.011455 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85534a2e-61d6-4f3f-acb5-d68a29ae1363 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:42.024207 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc353f30-2b46-43d6-a1c9-40071f0ef9bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:42.025232 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc353f30-2b46-43d6-a1c9-40071f0ef9bb admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:42.025741 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc353f30-2b46-43d6-a1c9-40071f0ef9bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:42.026173 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc353f30-2b46-43d6-a1c9-40071f0ef9bb admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:42.027223 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc353f30-2b46-43d6-a1c9-40071f0ef9bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:27:42.027711 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc353f30-2b46-43d6-a1c9-40071f0ef9bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:42.029548 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc353f30-2b46-43d6-a1c9-40071f0ef9bb admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:42.048355 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1430/2850] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 4 09:27:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:27:42.053730 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7a2dc7-c53e-4312-8afb-6fe1bded41d8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:42.065078 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7a2dc7-c53e-4312-8afb-6fe1bded41d8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:42.071436 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf0bfee-4121-49ea-93c8-703d7aebfd13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:42.072290 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf0bfee-4121-49ea-93c8-703d7aebfd13 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:42.072649 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf0bfee-4121-49ea-93c8-703d7aebfd13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:42.073001 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf0bfee-4121-49ea-93c8-703d7aebfd13 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:42.073842 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf0bfee-4121-49ea-93c8-703d7aebfd13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:42.075898 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf0bfee-4121-49ea-93c8-703d7aebfd13 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:42.117604 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1421/2851] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 4 09:27:42 2022] POST /identity/v3/projects => generated 375 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:27:42.119106 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc353f30-2b46-43d6-a1c9-40071f0ef9bb admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:42.129817 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc353f30-2b46-43d6-a1c9-40071f0ef9bb admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:42.206649 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351349c9-0a19-4a7d-ab89-db4ff60e633a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:42.207436 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351349c9-0a19-4a7d-ab89-db4ff60e633a admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:42.207796 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351349c9-0a19-4a7d-ab89-db4ff60e633a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:42.208143 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351349c9-0a19-4a7d-ab89-db4ff60e633a admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:42.209588 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351349c9-0a19-4a7d-ab89-db4ff60e633a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:42.211660 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351349c9-0a19-4a7d-ab89-db4ff60e633a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:42.248318 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1431/2852] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 4 09:27:42 2022] POST /identity/v3/users => generated 288 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:27:42.252302 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf0bfee-4121-49ea-93c8-703d7aebfd13 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:42.266319 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf0bfee-4121-49ea-93c8-703d7aebfd13 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:42.272932 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778fbed4-0c14-4d76-b438-455574dc657c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:42.273293 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778fbed4-0c14-4d76-b438-455574dc657c admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:42.273604 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778fbed4-0c14-4d76-b438-455574dc657c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:42.273892 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778fbed4-0c14-4d76-b438-455574dc657c admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:42.274968 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778fbed4-0c14-4d76-b438-455574dc657c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:42.277220 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778fbed4-0c14-4d76-b438-455574dc657c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:42.293433 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1422/2853] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:27:42 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:27:42.294679 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351349c9-0a19-4a7d-ab89-db4ff60e633a admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:42.306904 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351349c9-0a19-4a7d-ab89-db4ff60e633a admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:42.312315 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ba3b7b-08de-44a5-bc6b-e31c84d418b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:42.313176 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba3b7b-08de-44a5-bc6b-e31c84d418b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:42.313607 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba3b7b-08de-44a5-bc6b-e31c84d418b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:42.314062 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba3b7b-08de-44a5-bc6b-e31c84d418b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:42.314956 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ba3b7b-08de-44a5-bc6b-e31c84d418b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:42.321166 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ba3b7b-08de-44a5-bc6b-e31c84d418b4 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:42.333423 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1432/2854] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 4 09:27:42 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:27:42.338812 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778fbed4-0c14-4d76-b438-455574dc657c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:42.349669 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778fbed4-0c14-4d76-b438-455574dc657c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:42.355023 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc3fd9c-0bf8-4369-a851-3b2886a24a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:42.355913 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3fd9c-0bf8-4369-a851-3b2886a24a69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:42.356359 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3fd9c-0bf8-4369-a851-3b2886a24a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:42.356642 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3fd9c-0bf8-4369-a851-3b2886a24a69 admin admin] PATH_INFO: `/v3/projects/21ffc237d7f243dc9d0efe8a367b5077/users/21b2f682cc1847a180eafa93f9d90d35/roles/8692e99e9463459f9b7caa96b3051b93` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:42.373405 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc3fd9c-0bf8-4369-a851-3b2886a24a69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21ffc237d7f243dc9d0efe8a367b5077, user_id=21b2f682cc1847a180eafa93f9d90d35, role_id=8692e99e9463459f9b7caa96b3051b93)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:42.376541 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc3fd9c-0bf8-4369-a851-3b2886a24a69 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:42.396421 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1423/2855] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 4 09:27:42 2022] PUT /identity/v3/projects/21ffc237d7f243dc9d0efe8a367b5077/users/21b2f682cc1847a180eafa93f9d90d35/roles/8692e99e9463459f9b7caa96b3051b93 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:27:42.408664 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1725cf4f-3f3d-4c3e-9d38-3524fe6e6d8e None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:42.410477 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1725cf4f-3f3d-4c3e-9d38-3524fe6e6d8e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:42.410840 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1725cf4f-3f3d-4c3e-9d38-3524fe6e6d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:42.420041 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:27:42.449304 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-1725cf4f-3f3d-4c3e-9d38-3524fe6e6d8e None None] MFA Rules not processed for user `21b2f682cc1847a180eafa93f9d90d35`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:27:42.450634 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-1725cf4f-3f3d-4c3e-9d38-3524fe6e6d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:42.523226 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1433/2856] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:27:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:27:42.534957 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc3fd9c-0bf8-4369-a851-3b2886a24a69 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:42.537146 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5dc3fd9c-0bf8-4369-a851-3b2886a24a69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:42.635385 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc3fd9c-0bf8-4369-a851-3b2886a24a69 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:42.639192 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d2ab74-cc44-48d9-9800-ac55f91b352a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6e50a3715a14714b3951d36338dde42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:42.639856 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d2ab74-cc44-48d9-9800-ac55f91b352a service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:42.640142 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d2ab74-cc44-48d9-9800-ac55f91b352a service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:42.640420 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d2ab74-cc44-48d9-9800-ac55f91b352a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:42.651546 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d2ab74-cc44-48d9-9800-ac55f91b352a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:42.652593 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d2ab74-cc44-48d9-9800-ac55f91b352a service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:42.670967 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1424/2857] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:27:42 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:27:44.198858 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79245c6-a6cb-40a1-9480-15ffd3a22c44 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:44.199748 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79245c6-a6cb-40a1-9480-15ffd3a22c44 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:44.200233 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79245c6-a6cb-40a1-9480-15ffd3a22c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:44.233661 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-d79245c6-a6cb-40a1-9480-15ffd3a22c44 None None] MFA Rules not processed for user `21b2f682cc1847a180eafa93f9d90d35`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:27:44.235421 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d79245c6-a6cb-40a1-9480-15ffd3a22c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:44.247009 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1434/2858] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:27:44 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:27:46.847279 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d83ab7-7e4e-46b8-a1e9-4d6d1a106e74 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:46.847841 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d83ab7-7e4e-46b8-a1e9-4d6d1a106e74 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:46.847841 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d83ab7-7e4e-46b8-a1e9-4d6d1a106e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:46.889459 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-87d83ab7-7e4e-46b8-a1e9-4d6d1a106e74 None None] MFA Rules not processed for user `45747dabd8254bbea4dddd514647c4e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:27:46.890934 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-87d83ab7-7e4e-46b8-a1e9-4d6d1a106e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:47.003106 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:27:47.032424 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1425/2859] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:27:46 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:27:47.044260 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79245c6-a6cb-40a1-9480-15ffd3a22c44 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:47.047073 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d79245c6-a6cb-40a1-9480-15ffd3a22c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:47.182322 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79245c6-a6cb-40a1-9480-15ffd3a22c44 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:47.189031 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d587131d-7c45-4ad4-8c6b-ad56253a4f10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15039da209ec445d93d49e2bd6d854eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:47.189964 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587131d-7c45-4ad4-8c6b-ad56253a4f10 service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:47.190570 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587131d-7c45-4ad4-8c6b-ad56253a4f10 service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:47.191015 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587131d-7c45-4ad4-8c6b-ad56253a4f10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:47.194630 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d587131d-7c45-4ad4-8c6b-ad56253a4f10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:47.315042 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d587131d-7c45-4ad4-8c6b-ad56253a4f10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:47.316857 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d587131d-7c45-4ad4-8c6b-ad56253a4f10 service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:47.328423 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1435/2860] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:27:47 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:27:52.609128 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d83ab7-7e4e-46b8-a1e9-4d6d1a106e74 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:52.610582 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-87d83ab7-7e4e-46b8-a1e9-4d6d1a106e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:52.671930 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d83ab7-7e4e-46b8-a1e9-4d6d1a106e74 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:52.675484 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c5216b-0af1-4efe-84b7-b46167ae84d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:52.676136 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5216b-0af1-4efe-84b7-b46167ae84d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:52.676421 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5216b-0af1-4efe-84b7-b46167ae84d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:52.676697 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5216b-0af1-4efe-84b7-b46167ae84d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:52.678767 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-52c5216b-0af1-4efe-84b7-b46167ae84d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:52.689386 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c5216b-0af1-4efe-84b7-b46167ae84d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:52.691163 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c5216b-0af1-4efe-84b7-b46167ae84d0 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:52.700833 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1426/2861] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:27:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:27:57.323051 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:28:02.696795 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=29 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:28:02.757354 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c112aa0-10f2-4f49-9997-3394127561af None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:28:02.757842 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c112aa0-10f2-4f49-9997-3394127561af None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:28:02.757942 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c112aa0-10f2-4f49-9997-3394127561af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:28:02.777942 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-0c112aa0-10f2-4f49-9997-3394127561af None None] MFA Rules not processed for user `78668b939c5847b1b73e9c28b5d8db7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:28:02.779164 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0c112aa0-10f2-4f49-9997-3394127561af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:28:02.855325 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1436/2862] 10.222.0.71 () {58 vars in 1009 bytes} [Mon Jul 4 09:28:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:28:02.935359 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c5216b-0af1-4efe-84b7-b46167ae84d0 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:28:02.936862 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-52c5216b-0af1-4efe-84b7-b46167ae84d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:28:03.034217 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c5216b-0af1-4efe-84b7-b46167ae84d0 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:28:03.042092 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28872f18-bfa1-472f-a577-c93089c4dd44 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88b41b55769a4c449716c3590ed58225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:28:03.043101 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28872f18-bfa1-472f-a577-c93089c4dd44 service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:28:03.043468 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28872f18-bfa1-472f-a577-c93089c4dd44 service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:28:03.043932 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28872f18-bfa1-472f-a577-c93089c4dd44 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:28:03.055908 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28872f18-bfa1-472f-a577-c93089c4dd44 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:28:03.059042 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28872f18-bfa1-472f-a577-c93089c4dd44 service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:28:03.072206 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1427/2863] 10.222.0.71 () {60 vars in 1384 bytes} [Mon Jul 4 09:28:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 04 09:28:03.639229 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeaa7b7-529c-46dd-adad-b0a19e9b7bf7 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:28:03.639520 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeaa7b7-529c-46dd-adad-b0a19e9b7bf7 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:28:03.639801 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeaa7b7-529c-46dd-adad-b0a19e9b7bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:28:03.657391 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-edeaa7b7-529c-46dd-adad-b0a19e9b7bf7 None None] MFA Rules not processed for user `78668b939c5847b1b73e9c28b5d8db7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:28:03.658592 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-edeaa7b7-529c-46dd-adad-b0a19e9b7bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:28:03.665911 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1437/2864] 10.222.0.71 () {58 vars in 967 bytes} [Mon Jul 4 09:28:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:28:03.684615 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28872f18-bfa1-472f-a577-c93089c4dd44 service swift] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:28:03.696067 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28872f18-bfa1-472f-a577-c93089c4dd44 service swift] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:28:03.701116 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dac0f4d-7d3e-4cb6-a1fb-566b800dc974 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88b41b55769a4c449716c3590ed58225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:28:03.701788 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac0f4d-7d3e-4cb6-a1fb-566b800dc974 service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:28:03.702082 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac0f4d-7d3e-4cb6-a1fb-566b800dc974 service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:28:03.702366 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac0f4d-7d3e-4cb6-a1fb-566b800dc974 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:28:03.710573 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dac0f4d-7d3e-4cb6-a1fb-566b800dc974 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:28:03.712130 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dac0f4d-7d3e-4cb6-a1fb-566b800dc974 service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:28:03.720803 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1428/2865] 10.222.0.71 () {60 vars in 1384 bytes} [Mon Jul 4 09:28:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 09:28:03.917789 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b543de-354c-4199-bc3e-c22b5f5118ce None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:28:03.918331 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b543de-354c-4199-bc3e-c22b5f5118ce None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:28:03.918576 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b543de-354c-4199-bc3e-c22b5f5118ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:28:03.937957 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-35b543de-354c-4199-bc3e-c22b5f5118ce None None] MFA Rules not processed for user `78668b939c5847b1b73e9c28b5d8db7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:28:03.939188 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-35b543de-354c-4199-bc3e-c22b5f5118ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:28:03.946736 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1438/2866] 10.222.0.71 () {58 vars in 1009 bytes} [Mon Jul 4 09:28:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:28:03.958452 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dac0f4d-7d3e-4cb6-a1fb-566b800dc974 service swift] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:28:03.969781 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dac0f4d-7d3e-4cb6-a1fb-566b800dc974 service swift] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:28:03.976091 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17383c1c-48ed-4612-9c22-5617e68ad6ba service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88b41b55769a4c449716c3590ed58225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:28:03.976821 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17383c1c-48ed-4612-9c22-5617e68ad6ba service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:28:03.977212 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17383c1c-48ed-4612-9c22-5617e68ad6ba service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:28:03.977590 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17383c1c-48ed-4612-9c22-5617e68ad6ba service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:28:03.986375 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17383c1c-48ed-4612-9c22-5617e68ad6ba service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:28:03.988044 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17383c1c-48ed-4612-9c22-5617e68ad6ba service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:28:03.997257 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1429/2867] 10.222.0.71 () {60 vars in 1384 bytes} [Mon Jul 4 09:28:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 04 09:28:04.170619 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b543de-354c-4199-bc3e-c22b5f5118ce None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:28:04.172125 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-35b543de-354c-4199-bc3e-c22b5f5118ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:28:04.266051 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b543de-354c-4199-bc3e-c22b5f5118ce None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:28:04.271012 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f58b95f-ad33-4e13-8449-6d9265e7c0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:28:04.272544 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f58b95f-ad33-4e13-8449-6d9265e7c0d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:28:04.272892 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f58b95f-ad33-4e13-8449-6d9265e7c0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:28:04.273265 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f58b95f-ad33-4e13-8449-6d9265e7c0d3 admin admin] PATH_INFO: `/v3/users/aebfbcf03e2046ad918f84a6ee457dc4` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:28:04.274625 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f58b95f-ad33-4e13-8449-6d9265e7c0d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aebfbcf03e2046ad918f84a6ee457dc4)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:28:04.276690 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f58b95f-ad33-4e13-8449-6d9265e7c0d3 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:28:04.330079 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-8f58b95f-ad33-4e13-8449-6d9265e7c0d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aebfbcf03e2046ad918f84a6ee457dc4'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:28:04.341454 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-8f58b95f-ad33-4e13-8449-6d9265e7c0d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aebfbcf03e2046ad918f84a6ee457dc4'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:28:04.342246 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-8f58b95f-ad33-4e13-8449-6d9265e7c0d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aebfbcf03e2046ad918f84a6ee457dc4'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:28:04.352160 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-8f58b95f-ad33-4e13-8449-6d9265e7c0d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aebfbcf03e2046ad918f84a6ee457dc4'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:28:04.360130 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6958d1-5cc2-461f-9287-eac75bd73a86 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:28:04.360446 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6958d1-5cc2-461f-9287-eac75bd73a86 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:28:04.360737 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6958d1-5cc2-461f-9287-eac75bd73a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:28:04.379596 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-8f58b95f-ad33-4e13-8449-6d9265e7c0d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aebfbcf03e2046ad918f84a6ee457dc4'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:28:04.385957 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9b6958d1-5cc2-461f-9287-eac75bd73a86 None None] MFA Rules not processed for user `78668b939c5847b1b73e9c28b5d8db7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:28:04.387372 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9b6958d1-5cc2-461f-9287-eac75bd73a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:28:04.390487 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1439/2868] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:28:04 2022] DELETE /identity/v3/users/aebfbcf03e2046ad918f84a6ee457dc4 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:28:04.395026 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406cea5-1f0c-4207-ba93-d44e8a24344e None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:28:04.395323 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406cea5-1f0c-4207-ba93-d44e8a24344e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:28:04.395608 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406cea5-1f0c-4207-ba93-d44e8a24344e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:28:04.423804 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-1406cea5-1f0c-4207-ba93-d44e8a24344e None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:28:04.425186 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-1406cea5-1f0c-4207-ba93-d44e8a24344e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:28:04.430473 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:28:04.468194 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1430/2869] 10.222.0.71 () {58 vars in 967 bytes} [Mon Jul 4 09:28:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:28:04.481792 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6958d1-5cc2-461f-9287-eac75bd73a86 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:28:04.483493 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9b6958d1-5cc2-461f-9287-eac75bd73a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:28:04.498828 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1440/2870] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:28:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:28:04.509538 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1406cea5-1f0c-4207-ba93-d44e8a24344e None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:28:04.511271 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-1406cea5-1f0c-4207-ba93-d44e8a24344e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:28:04.570164 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6958d1-5cc2-461f-9287-eac75bd73a86 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:28:04.573338 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1406cea5-1f0c-4207-ba93-d44e8a24344e None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:28:04.576346 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb4a3db-43a4-4e2c-a89a-9cd8b2099881 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88b41b55769a4c449716c3590ed58225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:28:04.577008 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4a3db-43a4-4e2c-a89a-9cd8b2099881 service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:28:04.577286 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b5873e-929d-4979-863b-8f81e1822b87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:28:04.577434 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4a3db-43a4-4e2c-a89a-9cd8b2099881 service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:28:04.577587 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4a3db-43a4-4e2c-a89a-9cd8b2099881 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:28:04.578195 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5873e-929d-4979-863b-8f81e1822b87 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:28:04.578491 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5873e-929d-4979-863b-8f81e1822b87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:28:04.578774 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5873e-929d-4979-863b-8f81e1822b87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:28:04.588846 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb4a3db-43a4-4e2c-a89a-9cd8b2099881 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:28:04.589244 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b5873e-929d-4979-863b-8f81e1822b87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:28:04.591145 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b5873e-929d-4979-863b-8f81e1822b87 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:28:04.592242 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb4a3db-43a4-4e2c-a89a-9cd8b2099881 service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:28:04.597961 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:28:04.602791 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1441/2871] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:28:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:28:04.609374 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1431/2872] 10.222.0.71 () {60 vars in 1384 bytes} [Mon Jul 4 09:28:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 04 09:28:04.917467 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b5873e-929d-4979-863b-8f81e1822b87 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:28:04.919045 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-64b5873e-929d-4979-863b-8f81e1822b87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:28:04.934731 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b5873e-929d-4979-863b-8f81e1822b87 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:28:04.941737 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e33eb2-f097-4b16-92ab-1ef1d12c6ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:28:04.942683 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e33eb2-f097-4b16-92ab-1ef1d12c6ce7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:28:04.943107 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e33eb2-f097-4b16-92ab-1ef1d12c6ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:28:04.944117 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e33eb2-f097-4b16-92ab-1ef1d12c6ce7 admin admin] PATH_INFO: `/v3/projects/e1b430045396418d97cbc6b7dc9de645` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:28:04.945761 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e33eb2-f097-4b16-92ab-1ef1d12c6ce7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1b430045396418d97cbc6b7dc9de645)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:28:04.948276 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e33eb2-f097-4b16-92ab-1ef1d12c6ce7 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:28:05.005286 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-66e33eb2-f097-4b16-92ab-1ef1d12c6ce7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1b430045396418d97cbc6b7dc9de645', 'request_id': 'req-66e33eb2-f097-4b16-92ab-1ef1d12c6ce7'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:28:05.012352 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-66e33eb2-f097-4b16-92ab-1ef1d12c6ce7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1b430045396418d97cbc6b7dc9de645', 'request_id': 'req-66e33eb2-f097-4b16-92ab-1ef1d12c6ce7'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:28:05.020864 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-66e33eb2-f097-4b16-92ab-1ef1d12c6ce7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1b430045396418d97cbc6b7dc9de645', 'request_id': 'req-66e33eb2-f097-4b16-92ab-1ef1d12c6ce7'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:28:05.038101 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-66e33eb2-f097-4b16-92ab-1ef1d12c6ce7 admin admin] The token cache is being invalidate because project e1b430045396418d97cbc6b7dc9de645 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:28:05.038471 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-66e33eb2-f097-4b16-92ab-1ef1d12c6ce7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:28:05.039188 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-66e33eb2-f097-4b16-92ab-1ef1d12c6ce7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:28:05.041198 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1442/2873] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:28:04 2022] DELETE /identity/v3/projects/e1b430045396418d97cbc6b7dc9de645 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:28:06.618566 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e9ce04-8662-4f8b-9b97-3347002b2499 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:28:06.619404 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e9ce04-8662-4f8b-9b97-3347002b2499 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:28:06.619896 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e9ce04-8662-4f8b-9b97-3347002b2499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:28:06.647296 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-72e9ce04-8662-4f8b-9b97-3347002b2499 None None] MFA Rules not processed for user `70733d7f6b8b42a39425ebc94e1ec1f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:28:06.648546 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-72e9ce04-8662-4f8b-9b97-3347002b2499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:28:06.713699 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1432/2874] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:28:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:28:06.753908 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e33eb2-f097-4b16-92ab-1ef1d12c6ce7 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:28:06.755614 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-66e33eb2-f097-4b16-92ab-1ef1d12c6ce7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:28:06.800987 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:28:06.830288 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e33eb2-f097-4b16-92ab-1ef1d12c6ce7 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:28:06.835431 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e3533f-336b-4393-8db2-9b42c30c75c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:28:06.836344 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e3533f-336b-4393-8db2-9b42c30c75c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:28:06.836636 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e3533f-336b-4393-8db2-9b42c30c75c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:28:06.836922 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e3533f-336b-4393-8db2-9b42c30c75c0 admin admin] PATH_INFO: `/v3/users/21b2f682cc1847a180eafa93f9d90d35` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:28:06.838274 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e3533f-336b-4393-8db2-9b42c30c75c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21b2f682cc1847a180eafa93f9d90d35)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:28:06.840226 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e3533f-336b-4393-8db2-9b42c30c75c0 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:28:06.895216 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e6e3533f-336b-4393-8db2-9b42c30c75c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21b2f682cc1847a180eafa93f9d90d35'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:28:06.904027 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e6e3533f-336b-4393-8db2-9b42c30c75c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21b2f682cc1847a180eafa93f9d90d35'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:28:06.904690 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e6e3533f-336b-4393-8db2-9b42c30c75c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21b2f682cc1847a180eafa93f9d90d35'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:28:06.912654 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e6e3533f-336b-4393-8db2-9b42c30c75c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21b2f682cc1847a180eafa93f9d90d35'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:28:06.925173 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e6e3533f-336b-4393-8db2-9b42c30c75c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21b2f682cc1847a180eafa93f9d90d35'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:28:06.932185 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1443/2875] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:28:06 2022] DELETE /identity/v3/users/21b2f682cc1847a180eafa93f9d90d35 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:28:06.936650 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d40e8e1-6a47-4a87-a11c-3f488240100f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:28:06.937178 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d40e8e1-6a47-4a87-a11c-3f488240100f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:28:06.937464 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d40e8e1-6a47-4a87-a11c-3f488240100f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:28:06.954896 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6d40e8e1-6a47-4a87-a11c-3f488240100f None None] MFA Rules not processed for user `fa43c0e65110476d9e52a93b45c1a988`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:28:06.956139 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6d40e8e1-6a47-4a87-a11c-3f488240100f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:28:07.019567 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1433/2876] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:28:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:28:07.029426 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e3533f-336b-4393-8db2-9b42c30c75c0 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:28:07.031177 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e6e3533f-336b-4393-8db2-9b42c30c75c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:28:07.091026 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e3533f-336b-4393-8db2-9b42c30c75c0 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:28:07.095478 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922d7307-7a3b-41f4-83aa-696e2dcb59c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:28:07.096138 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d7307-7a3b-41f4-83aa-696e2dcb59c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:28:07.096429 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d7307-7a3b-41f4-83aa-696e2dcb59c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:28:07.096713 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d7307-7a3b-41f4-83aa-696e2dcb59c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:28:07.106763 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922d7307-7a3b-41f4-83aa-696e2dcb59c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:28:07.108219 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922d7307-7a3b-41f4-83aa-696e2dcb59c1 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:28:07.118451 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1444/2877] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:28:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:28:07.298128 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d40e8e1-6a47-4a87-a11c-3f488240100f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:28:07.300496 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6d40e8e1-6a47-4a87-a11c-3f488240100f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:28:07.313778 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d40e8e1-6a47-4a87-a11c-3f488240100f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:28:07.318833 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4576741b-db7e-4290-8cb3-b671ed9c0835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:28:07.320746 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4576741b-db7e-4290-8cb3-b671ed9c0835 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:28:07.321250 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4576741b-db7e-4290-8cb3-b671ed9c0835 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:28:07.321914 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4576741b-db7e-4290-8cb3-b671ed9c0835 admin admin] PATH_INFO: `/v3/projects/21ffc237d7f243dc9d0efe8a367b5077` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:28:07.323417 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4576741b-db7e-4290-8cb3-b671ed9c0835 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21ffc237d7f243dc9d0efe8a367b5077)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:28:07.326121 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4576741b-db7e-4290-8cb3-b671ed9c0835 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:28:07.391284 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4576741b-db7e-4290-8cb3-b671ed9c0835 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21ffc237d7f243dc9d0efe8a367b5077', 'request_id': 'req-4576741b-db7e-4290-8cb3-b671ed9c0835'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:28:07.399429 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4576741b-db7e-4290-8cb3-b671ed9c0835 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21ffc237d7f243dc9d0efe8a367b5077', 'request_id': 'req-4576741b-db7e-4290-8cb3-b671ed9c0835'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:28:07.407531 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4576741b-db7e-4290-8cb3-b671ed9c0835 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21ffc237d7f243dc9d0efe8a367b5077', 'request_id': 'req-4576741b-db7e-4290-8cb3-b671ed9c0835'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:28:07.422242 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4576741b-db7e-4290-8cb3-b671ed9c0835 admin admin] The token cache is being invalidate because project 21ffc237d7f243dc9d0efe8a367b5077 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:28:07.422538 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4576741b-db7e-4290-8cb3-b671ed9c0835 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:28:07.423190 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4576741b-db7e-4290-8cb3-b671ed9c0835 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:28:07.424767 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1434/2878] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:28:07 2022] DELETE /identity/v3/projects/21ffc237d7f243dc9d0efe8a367b5077 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:28:09.336280 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922d7307-7a3b-41f4-83aa-696e2dcb59c1 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:28:09.338370 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-922d7307-7a3b-41f4-83aa-696e2dcb59c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:28:09.351870 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922d7307-7a3b-41f4-83aa-696e2dcb59c1 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:28:09.422353 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58043986-74f7-4132-9553-b7d1b1db6d60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:28:09.423025 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58043986-74f7-4132-9553-b7d1b1db6d60 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:28:09.423312 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58043986-74f7-4132-9553-b7d1b1db6d60 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:28:09.423733 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58043986-74f7-4132-9553-b7d1b1db6d60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:28:09.427570 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-58043986-74f7-4132-9553-b7d1b1db6d60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:28:09.449624 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:28:09.527780 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58043986-74f7-4132-9553-b7d1b1db6d60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:28:09.529409 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58043986-74f7-4132-9553-b7d1b1db6d60 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:28:09.550257 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1445/2879] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:28:09 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:28:17.417861 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=80,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:28:19.535719 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=23 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:28:23.323270 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4576741b-db7e-4290-8cb3-b671ed9c0835 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:28:23.324744 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4576741b-db7e-4290-8cb3-b671ed9c0835 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:28:23.338189 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4576741b-db7e-4290-8cb3-b671ed9c0835 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:28:23.341855 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2097f9c7-4028-4346-a0f2-bb82b8ce36e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:28:23.342500 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2097f9c7-4028-4346-a0f2-bb82b8ce36e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:28:23.342781 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2097f9c7-4028-4346-a0f2-bb82b8ce36e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:28:23.343057 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2097f9c7-4028-4346-a0f2-bb82b8ce36e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:28:23.344768 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2097f9c7-4028-4346-a0f2-bb82b8ce36e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:28:23.408402 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2097f9c7-4028-4346-a0f2-bb82b8ce36e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:28:23.409846 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2097f9c7-4028-4346-a0f2-bb82b8ce36e0 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:28:23.419341 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1435/2880] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:28:23 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:28:33.415160 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=30 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:30:09.363710 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58043986-74f7-4132-9553-b7d1b1db6d60 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:09.375037 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58043986-74f7-4132-9553-b7d1b1db6d60 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:09.378830 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6242dc-bc60-403d-9a2a-9a66bb37302b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:09.379478 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6242dc-bc60-403d-9a2a-9a66bb37302b service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:09.379763 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6242dc-bc60-403d-9a2a-9a66bb37302b service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:09.380040 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6242dc-bc60-403d-9a2a-9a66bb37302b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:09.381896 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-6e6242dc-bc60-403d-9a2a-9a66bb37302b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:30:09.456709 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6242dc-bc60-403d-9a2a-9a66bb37302b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:09.458278 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6242dc-bc60-403d-9a2a-9a66bb37302b service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:09.470032 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1446/2881] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:30:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:30:10.656058 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2097f9c7-4028-4346-a0f2-bb82b8ce36e0 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:10.670916 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2097f9c7-4028-4346-a0f2-bb82b8ce36e0 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:10.675489 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2c6c45-e7bc-4365-b145-c9720675a637 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:10.676149 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c6c45-e7bc-4365-b145-c9720675a637 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:10.676435 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c6c45-e7bc-4365-b145-c9720675a637 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:10.676715 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c6c45-e7bc-4365-b145-c9720675a637 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:10.678549 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4a2c6c45-e7bc-4365-b145-c9720675a637 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:30:10.694293 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2c6c45-e7bc-4365-b145-c9720675a637 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:10.695725 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2c6c45-e7bc-4365-b145-c9720675a637 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:10.709564 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1436/2882] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:30:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:30:19.464881 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=34 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:30:19.498431 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6242dc-bc60-403d-9a2a-9a66bb37302b service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:19.499864 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-6e6242dc-bc60-403d-9a2a-9a66bb37302b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:30:19.512937 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6242dc-bc60-403d-9a2a-9a66bb37302b service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:19.517056 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9cdd7c-e4d7-4cb5-b2c3-ce920bb4a178 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:19.517810 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9cdd7c-e4d7-4cb5-b2c3-ce920bb4a178 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:19.518106 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9cdd7c-e4d7-4cb5-b2c3-ce920bb4a178 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:19.518396 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9cdd7c-e4d7-4cb5-b2c3-ce920bb4a178 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:19.520325 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-dd9cdd7c-e4d7-4cb5-b2c3-ce920bb4a178 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:30:19.531199 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9cdd7c-e4d7-4cb5-b2c3-ce920bb4a178 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:19.532700 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9cdd7c-e4d7-4cb5-b2c3-ce920bb4a178 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:19.541091 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1447/2883] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:30:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:30:20.703080 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=6 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:30:22.774045 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2c6c45-e7bc-4365-b145-c9720675a637 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:22.783685 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2c6c45-e7bc-4365-b145-c9720675a637 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:22.787322 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2b6113-79bd-4e68-a92a-f087a9596edb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:22.787961 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b6113-79bd-4e68-a92a-f087a9596edb service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:22.788243 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b6113-79bd-4e68-a92a-f087a9596edb service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:22.788521 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b6113-79bd-4e68-a92a-f087a9596edb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:22.790320 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1f2b6113-79bd-4e68-a92a-f087a9596edb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:30:22.801553 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2b6113-79bd-4e68-a92a-f087a9596edb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:22.802969 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2b6113-79bd-4e68-a92a-f087a9596edb service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:22.812606 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1437/2884] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:30:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:30:22.892742 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9cdd7c-e4d7-4cb5-b2c3-ce920bb4a178 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:22.901952 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9cdd7c-e4d7-4cb5-b2c3-ce920bb4a178 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:22.905866 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b15b25-3279-423b-8cda-20760c8529fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:22.906651 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b15b25-3279-423b-8cda-20760c8529fd service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:22.907053 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b15b25-3279-423b-8cda-20760c8529fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:22.907295 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b15b25-3279-423b-8cda-20760c8529fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:22.909186 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d2b15b25-3279-423b-8cda-20760c8529fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:30:22.924729 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b15b25-3279-423b-8cda-20760c8529fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:22.926206 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b15b25-3279-423b-8cda-20760c8529fd service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:22.938238 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1448/2885] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:30:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:30:22.998515 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2b6113-79bd-4e68-a92a-f087a9596edb service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:22.999895 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1f2b6113-79bd-4e68-a92a-f087a9596edb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:30:23.070355 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2b6113-79bd-4e68-a92a-f087a9596edb service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:23.075983 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fd8fbe-7b0a-4e66-a1e6-4f59a6dd9b94 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '293c695f15c8437d81a619f840318f44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:23.076638 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fd8fbe-7b0a-4e66-a1e6-4f59a6dd9b94 service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:23.076922 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fd8fbe-7b0a-4e66-a1e6-4f59a6dd9b94 service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:23.077231 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fd8fbe-7b0a-4e66-a1e6-4f59a6dd9b94 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:23.079065 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a9fd8fbe-7b0a-4e66-a1e6-4f59a6dd9b94 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:30:23.090738 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fd8fbe-7b0a-4e66-a1e6-4f59a6dd9b94 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:23.092290 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fd8fbe-7b0a-4e66-a1e6-4f59a6dd9b94 service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:23.102237 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1438/2886] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:30:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:30:25.058658 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b15b25-3279-423b-8cda-20760c8529fd service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:25.060726 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d2b15b25-3279-423b-8cda-20760c8529fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:30:25.083724 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b15b25-3279-423b-8cda-20760c8529fd service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:25.092200 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d79c43a-d452-4166-bb7a-eb0f548fb5a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15039da209ec445d93d49e2bd6d854eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:25.093117 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d79c43a-d452-4166-bb7a-eb0f548fb5a7 service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:25.093585 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d79c43a-d452-4166-bb7a-eb0f548fb5a7 service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:25.094034 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d79c43a-d452-4166-bb7a-eb0f548fb5a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:25.096761 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-8d79c43a-d452-4166-bb7a-eb0f548fb5a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:30:25.129350 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d79c43a-d452-4166-bb7a-eb0f548fb5a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:25.132339 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d79c43a-d452-4166-bb7a-eb0f548fb5a7 service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:25.148570 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1449/2887] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:30:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:30:25.368065 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fd8fbe-7b0a-4e66-a1e6-4f59a6dd9b94 service placement] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:25.391263 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fd8fbe-7b0a-4e66-a1e6-4f59a6dd9b94 service placement] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:25.404006 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2af4c2-1d32-4bae-8e83-714113027af1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15039da209ec445d93d49e2bd6d854eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:25.404678 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2af4c2-1d32-4bae-8e83-714113027af1 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:25.405193 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2af4c2-1d32-4bae-8e83-714113027af1 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:25.405552 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2af4c2-1d32-4bae-8e83-714113027af1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:25.407492 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3f2af4c2-1d32-4bae-8e83-714113027af1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:30:25.422060 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2af4c2-1d32-4bae-8e83-714113027af1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:25.423836 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2af4c2-1d32-4bae-8e83-714113027af1 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:25.442536 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1439/2888] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:30:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:30:27.132133 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702f460-6608-4f59-be90-1346ead69a5d None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:27.132661 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702f460-6608-4f59-be90-1346ead69a5d None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:27.132661 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702f460-6608-4f59-be90-1346ead69a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:27.150747 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-d702f460-6608-4f59-be90-1346ead69a5d None None] MFA Rules not processed for user `70733d7f6b8b42a39425ebc94e1ec1f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:30:27.151978 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d702f460-6608-4f59-be90-1346ead69a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:30:27.207405 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1450/2889] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:30:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:30:27.811134 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2af4c2-1d32-4bae-8e83-714113027af1 service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:27.820755 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2af4c2-1d32-4bae-8e83-714113027af1 service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:27.824918 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca1e566-abe2-47da-a222-c3a7b1399539 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:27.825659 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca1e566-abe2-47da-a222-c3a7b1399539 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:27.825947 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca1e566-abe2-47da-a222-c3a7b1399539 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:27.826226 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca1e566-abe2-47da-a222-c3a7b1399539 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:27.828023 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2ca1e566-abe2-47da-a222-c3a7b1399539 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:30:27.838555 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca1e566-abe2-47da-a222-c3a7b1399539 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:27.840112 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca1e566-abe2-47da-a222-c3a7b1399539 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:27.848521 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1440/2890] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:30:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:30:37.200539 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=43 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:30:37.844635 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=50 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:31:51.467892 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d702f460-6608-4f59-be90-1346ead69a5d None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:31:51.470137 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d702f460-6608-4f59-be90-1346ead69a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:31:51.552124 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d702f460-6608-4f59-be90-1346ead69a5d None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:31:51.559008 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bc3b63-cdd1-44f9-a931-a05cd516340a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15039da209ec445d93d49e2bd6d854eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:31:51.559958 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc3b63-cdd1-44f9-a931-a05cd516340a service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:31:51.560468 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc3b63-cdd1-44f9-a931-a05cd516340a service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:31:51.560959 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc3b63-cdd1-44f9-a931-a05cd516340a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:31:51.563027 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-90bc3b63-cdd1-44f9-a931-a05cd516340a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:31:51.576419 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bc3b63-cdd1-44f9-a931-a05cd516340a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:31:51.577709 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bc3b63-cdd1-44f9-a931-a05cd516340a service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:31:51.589465 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1451/2891] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:31:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:31:58.095108 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca1e566-abe2-47da-a222-c3a7b1399539 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:31:58.096512 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2ca1e566-abe2-47da-a222-c3a7b1399539 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:31:58.146773 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca1e566-abe2-47da-a222-c3a7b1399539 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:31:58.152977 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9128098c-c688-4d36-bd40-29007da316ba service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '293c695f15c8437d81a619f840318f44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:31:58.153550 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128098c-c688-4d36-bd40-29007da316ba service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:31:58.153838 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128098c-c688-4d36-bd40-29007da316ba service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:31:58.154115 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128098c-c688-4d36-bd40-29007da316ba service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:31:58.155912 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9128098c-c688-4d36-bd40-29007da316ba service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:31:58.170509 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9128098c-c688-4d36-bd40-29007da316ba service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:31:58.172205 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9128098c-c688-4d36-bd40-29007da316ba service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:31:58.185214 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1441/2892] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:31:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:32:01.445693 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bc3b63-cdd1-44f9-a931-a05cd516340a service nova] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:32:01.457927 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bc3b63-cdd1-44f9-a931-a05cd516340a service nova] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:32:01.463570 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d542c83-8b7f-472f-84b7-8be7758e8adc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '293c695f15c8437d81a619f840318f44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:32:01.464257 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d542c83-8b7f-472f-84b7-8be7758e8adc service placement] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:32:01.464546 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d542c83-8b7f-472f-84b7-8be7758e8adc service placement] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:32:01.464837 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d542c83-8b7f-472f-84b7-8be7758e8adc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:32:01.467018 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-3d542c83-8b7f-472f-84b7-8be7758e8adc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:32:01.479995 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d542c83-8b7f-472f-84b7-8be7758e8adc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:32:01.482053 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d542c83-8b7f-472f-84b7-8be7758e8adc service placement] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:32:01.495306 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1452/2893] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:32:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:32:01.506876 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9128098c-c688-4d36-bd40-29007da316ba service placement] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:32:01.508319 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9128098c-c688-4d36-bd40-29007da316ba service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:32:01.623437 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9128098c-c688-4d36-bd40-29007da316ba service placement] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:32:01.628158 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6421fb6d-f34f-418b-9936-a67ddcd66c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:32:01.628982 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6421fb6d-f34f-418b-9936-a67ddcd66c06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:32:01.629298 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6421fb6d-f34f-418b-9936-a67ddcd66c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:32:01.629580 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6421fb6d-f34f-418b-9936-a67ddcd66c06 admin admin] PATH_INFO: `/v3/users/df06c3b8aef249b984bc907b1753747c` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:32:01.630777 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6421fb6d-f34f-418b-9936-a67ddcd66c06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df06c3b8aef249b984bc907b1753747c)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:32:01.632822 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6421fb6d-f34f-418b-9936-a67ddcd66c06 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:32:01.680993 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6421fb6d-f34f-418b-9936-a67ddcd66c06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df06c3b8aef249b984bc907b1753747c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:32:01.681609 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6421fb6d-f34f-418b-9936-a67ddcd66c06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df06c3b8aef249b984bc907b1753747c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:32:01.688474 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6421fb6d-f34f-418b-9936-a67ddcd66c06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df06c3b8aef249b984bc907b1753747c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:32:01.700688 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6421fb6d-f34f-418b-9936-a67ddcd66c06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df06c3b8aef249b984bc907b1753747c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:32:01.701346 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6421fb6d-f34f-418b-9936-a67ddcd66c06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df06c3b8aef249b984bc907b1753747c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:32:01.714437 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1442/2894] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:32:01 2022] DELETE /identity/v3/users/df06c3b8aef249b984bc907b1753747c => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:32:01.893725 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d542c83-8b7f-472f-84b7-8be7758e8adc service placement] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:32:01.909235 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d542c83-8b7f-472f-84b7-8be7758e8adc service placement] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:32:01.967618 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847055b7-7ee1-480e-b119-3a0fdb58b536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:32:01.968491 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847055b7-7ee1-480e-b119-3a0fdb58b536 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:32:01.968780 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847055b7-7ee1-480e-b119-3a0fdb58b536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:32:01.969091 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847055b7-7ee1-480e-b119-3a0fdb58b536 admin admin] PATH_INFO: `/v3/projects/a2b9fbff141540f0858c09f7ea1e80b9` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:32:01.970280 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847055b7-7ee1-480e-b119-3a0fdb58b536 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2b9fbff141540f0858c09f7ea1e80b9)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:32:01.972221 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847055b7-7ee1-480e-b119-3a0fdb58b536 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:32:02.018294 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-847055b7-7ee1-480e-b119-3a0fdb58b536 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2b9fbff141540f0858c09f7ea1e80b9', 'request_id': 'req-847055b7-7ee1-480e-b119-3a0fdb58b536'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:32:02.024459 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-847055b7-7ee1-480e-b119-3a0fdb58b536 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2b9fbff141540f0858c09f7ea1e80b9', 'request_id': 'req-847055b7-7ee1-480e-b119-3a0fdb58b536'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:32:02.030589 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-847055b7-7ee1-480e-b119-3a0fdb58b536 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2b9fbff141540f0858c09f7ea1e80b9', 'request_id': 'req-847055b7-7ee1-480e-b119-3a0fdb58b536'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:32:02.043228 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-847055b7-7ee1-480e-b119-3a0fdb58b536 admin admin] The token cache is being invalidate because project a2b9fbff141540f0858c09f7ea1e80b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:32:02.043525 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-847055b7-7ee1-480e-b119-3a0fdb58b536 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:32:02.044155 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-847055b7-7ee1-480e-b119-3a0fdb58b536 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:32:02.045744 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1453/2895] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:32:01 2022] DELETE /identity/v3/projects/a2b9fbff141540f0858c09f7ea1e80b9 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:32:02.049717 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6421fb6d-f34f-418b-9936-a67ddcd66c06 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:32:02.051124 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6421fb6d-f34f-418b-9936-a67ddcd66c06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:32:02.064281 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6421fb6d-f34f-418b-9936-a67ddcd66c06 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:32:02.069305 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:32:02.123309 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3e4760-09a9-4fa6-a2bc-5d70c9b46b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:32:02.124180 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3e4760-09a9-4fa6-a2bc-5d70c9b46b2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:32:02.124467 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3e4760-09a9-4fa6-a2bc-5d70c9b46b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:32:02.124803 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3e4760-09a9-4fa6-a2bc-5d70c9b46b2d admin admin] PATH_INFO: `/v3/users/7fd3ab8ccfda463f9eed5641141e0c87` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:32:02.125953 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3e4760-09a9-4fa6-a2bc-5d70c9b46b2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fd3ab8ccfda463f9eed5641141e0c87)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:32:02.127898 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3e4760-09a9-4fa6-a2bc-5d70c9b46b2d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:32:02.173860 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5f3e4760-09a9-4fa6-a2bc-5d70c9b46b2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fd3ab8ccfda463f9eed5641141e0c87'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:32:02.174515 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5f3e4760-09a9-4fa6-a2bc-5d70c9b46b2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fd3ab8ccfda463f9eed5641141e0c87'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:32:02.181658 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5f3e4760-09a9-4fa6-a2bc-5d70c9b46b2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fd3ab8ccfda463f9eed5641141e0c87'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:32:02.193377 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5f3e4760-09a9-4fa6-a2bc-5d70c9b46b2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fd3ab8ccfda463f9eed5641141e0c87'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:32:02.194545 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5f3e4760-09a9-4fa6-a2bc-5d70c9b46b2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fd3ab8ccfda463f9eed5641141e0c87'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:32:02.206643 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1443/2896] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:32:02 2022] DELETE /identity/v3/users/7fd3ab8ccfda463f9eed5641141e0c87 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:32:02.380408 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847055b7-7ee1-480e-b119-3a0fdb58b536 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:32:02.382444 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-847055b7-7ee1-480e-b119-3a0fdb58b536 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:32:02.402068 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847055b7-7ee1-480e-b119-3a0fdb58b536 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:32:02.416029 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:32:02.463033 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fc1a3d-c416-4c45-9f51-a29fc40a3691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:32:02.463934 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fc1a3d-c416-4c45-9f51-a29fc40a3691 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:32:02.464224 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fc1a3d-c416-4c45-9f51-a29fc40a3691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:32:02.464508 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fc1a3d-c416-4c45-9f51-a29fc40a3691 admin admin] PATH_INFO: `/v3/projects/5b984458174d4101bae3cff412ff7ef2` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:32:02.465738 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fc1a3d-c416-4c45-9f51-a29fc40a3691 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b984458174d4101bae3cff412ff7ef2)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:32:02.467910 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fc1a3d-c416-4c45-9f51-a29fc40a3691 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:32:02.516085 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-63fc1a3d-c416-4c45-9f51-a29fc40a3691 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b984458174d4101bae3cff412ff7ef2', 'request_id': 'req-63fc1a3d-c416-4c45-9f51-a29fc40a3691'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:32:02.521460 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-63fc1a3d-c416-4c45-9f51-a29fc40a3691 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b984458174d4101bae3cff412ff7ef2', 'request_id': 'req-63fc1a3d-c416-4c45-9f51-a29fc40a3691'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:32:02.527578 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-63fc1a3d-c416-4c45-9f51-a29fc40a3691 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b984458174d4101bae3cff412ff7ef2', 'request_id': 'req-63fc1a3d-c416-4c45-9f51-a29fc40a3691'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:32:02.540308 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-63fc1a3d-c416-4c45-9f51-a29fc40a3691 admin admin] The token cache is being invalidate because project 5b984458174d4101bae3cff412ff7ef2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:32:02.540600 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-63fc1a3d-c416-4c45-9f51-a29fc40a3691 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:32:02.541420 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-63fc1a3d-c416-4c45-9f51-a29fc40a3691 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:32:02.543153 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1454/2897] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:32:02 2022] DELETE /identity/v3/projects/5b984458174d4101bae3cff412ff7ef2 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:32:12.203182 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=49,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:32:12.538143 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=44,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:32:15.829633 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3e4760-09a9-4fa6-a2bc-5d70c9b46b2d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:32:15.831085 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5f3e4760-09a9-4fa6-a2bc-5d70c9b46b2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:32:15.931156 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3e4760-09a9-4fa6-a2bc-5d70c9b46b2d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:32:15.994176 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11bd53f-cf9b-456c-b849-8dad9777f637 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '293c695f15c8437d81a619f840318f44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:32:15.994837 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11bd53f-cf9b-456c-b849-8dad9777f637 service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:32:15.995125 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11bd53f-cf9b-456c-b849-8dad9777f637 service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:32:15.995423 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11bd53f-cf9b-456c-b849-8dad9777f637 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:32:15.997889 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d11bd53f-cf9b-456c-b849-8dad9777f637 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:32:16.011098 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11bd53f-cf9b-456c-b849-8dad9777f637 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:32:16.012666 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11bd53f-cf9b-456c-b849-8dad9777f637 service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:32:16.023605 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1444/2898] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:32:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:32:19.355876 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d293f11-5755-4256-aec9-f9cb88aa9e27 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:32:19.356386 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d293f11-5755-4256-aec9-f9cb88aa9e27 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:32:19.356498 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d293f11-5755-4256-aec9-f9cb88aa9e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:32:19.374766 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-3d293f11-5755-4256-aec9-f9cb88aa9e27 None None] MFA Rules not processed for user `70733d7f6b8b42a39425ebc94e1ec1f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:32:19.375985 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-3d293f11-5755-4256-aec9-f9cb88aa9e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:32:19.430797 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1455/2899] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 4 09:32:19 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:32:26.018395 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=67 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:32:29.423148 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=25 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:34:04.201549 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11bd53f-cf9b-456c-b849-8dad9777f637 service placement] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:34:04.203000 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d11bd53f-cf9b-456c-b849-8dad9777f637 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:34:04.282341 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11bd53f-cf9b-456c-b849-8dad9777f637 service placement] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:34:04.285991 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412ad6ea-fc23-40c5-b793-5acfa86ef3b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:34:04.286646 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ad6ea-fc23-40c5-b793-5acfa86ef3b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:34:04.287044 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ad6ea-fc23-40c5-b793-5acfa86ef3b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:34:04.287329 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ad6ea-fc23-40c5-b793-5acfa86ef3b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:34:04.289132 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-412ad6ea-fc23-40c5-b793-5acfa86ef3b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:34:04.299757 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412ad6ea-fc23-40c5-b793-5acfa86ef3b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:34:04.301208 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412ad6ea-fc23-40c5-b793-5acfa86ef3b6 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:34:04.310659 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1445/2900] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:34:04 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:34:14.306753 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=35 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:34:17.357483 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d293f11-5755-4256-aec9-f9cb88aa9e27 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:34:17.367500 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d293f11-5755-4256-aec9-f9cb88aa9e27 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:34:17.371244 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01cd16a-a8cd-4153-b7ce-6fd189ee5098 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15818e58661646f381da0973d4c2ff64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:34:17.371916 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01cd16a-a8cd-4153-b7ce-6fd189ee5098 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:34:17.372185 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01cd16a-a8cd-4153-b7ce-6fd189ee5098 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:34:17.372466 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01cd16a-a8cd-4153-b7ce-6fd189ee5098 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:34:17.374262 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a01cd16a-a8cd-4153-b7ce-6fd189ee5098 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:34:17.440747 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01cd16a-a8cd-4153-b7ce-6fd189ee5098 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:34:17.442742 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01cd16a-a8cd-4153-b7ce-6fd189ee5098 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:34:17.453701 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1456/2901] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:34:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:34:23.706388 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412ad6ea-fc23-40c5-b793-5acfa86ef3b6 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:34:23.708852 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-412ad6ea-fc23-40c5-b793-5acfa86ef3b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:34:23.732333 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412ad6ea-fc23-40c5-b793-5acfa86ef3b6 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:34:23.738511 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e121eb0-1b78-4f78-b4f5-ddf316d3ecf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:34:23.740143 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e121eb0-1b78-4f78-b4f5-ddf316d3ecf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:34:23.740404 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e121eb0-1b78-4f78-b4f5-ddf316d3ecf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:34:23.740741 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e121eb0-1b78-4f78-b4f5-ddf316d3ecf1 admin admin] PATH_INFO: `/v3/users/70733d7f6b8b42a39425ebc94e1ec1f4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:34:23.745457 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e121eb0-1b78-4f78-b4f5-ddf316d3ecf1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70733d7f6b8b42a39425ebc94e1ec1f4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:34:23.747683 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e121eb0-1b78-4f78-b4f5-ddf316d3ecf1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:34:23.806241 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8e121eb0-1b78-4f78-b4f5-ddf316d3ecf1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70733d7f6b8b42a39425ebc94e1ec1f4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:34:23.806963 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8e121eb0-1b78-4f78-b4f5-ddf316d3ecf1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70733d7f6b8b42a39425ebc94e1ec1f4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:34:23.813979 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8e121eb0-1b78-4f78-b4f5-ddf316d3ecf1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70733d7f6b8b42a39425ebc94e1ec1f4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:34:23.827639 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8e121eb0-1b78-4f78-b4f5-ddf316d3ecf1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70733d7f6b8b42a39425ebc94e1ec1f4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:34:23.828397 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8e121eb0-1b78-4f78-b4f5-ddf316d3ecf1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70733d7f6b8b42a39425ebc94e1ec1f4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:34:23.841846 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1446/2902] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:34:23 2022] DELETE /identity/v3/users/70733d7f6b8b42a39425ebc94e1ec1f4 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:34:24.033712 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01cd16a-a8cd-4153-b7ce-6fd189ee5098 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:34:24.045307 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01cd16a-a8cd-4153-b7ce-6fd189ee5098 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:34:24.105295 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735b397d-7536-4019-9f82-84a297c99645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:34:24.106219 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735b397d-7536-4019-9f82-84a297c99645 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:34:24.106507 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735b397d-7536-4019-9f82-84a297c99645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:34:24.106810 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735b397d-7536-4019-9f82-84a297c99645 admin admin] PATH_INFO: `/v3/projects/4fe236d825ae40f793bf2a5bccedc3cd` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:34:24.108030 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735b397d-7536-4019-9f82-84a297c99645 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fe236d825ae40f793bf2a5bccedc3cd)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:34:24.110003 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735b397d-7536-4019-9f82-84a297c99645 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:34:24.155263 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-735b397d-7536-4019-9f82-84a297c99645 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fe236d825ae40f793bf2a5bccedc3cd', 'request_id': 'req-735b397d-7536-4019-9f82-84a297c99645'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:34:24.160618 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-735b397d-7536-4019-9f82-84a297c99645 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fe236d825ae40f793bf2a5bccedc3cd', 'request_id': 'req-735b397d-7536-4019-9f82-84a297c99645'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:34:24.166828 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-735b397d-7536-4019-9f82-84a297c99645 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fe236d825ae40f793bf2a5bccedc3cd', 'request_id': 'req-735b397d-7536-4019-9f82-84a297c99645'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:34:24.179106 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-735b397d-7536-4019-9f82-84a297c99645 admin admin] The token cache is being invalidate because project 4fe236d825ae40f793bf2a5bccedc3cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:34:24.179398 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-735b397d-7536-4019-9f82-84a297c99645 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:34:24.180036 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-735b397d-7536-4019-9f82-84a297c99645 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:34:24.181617 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1457/2903] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:34:24 2022] DELETE /identity/v3/projects/4fe236d825ae40f793bf2a5bccedc3cd => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:34:24.185589 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e121eb0-1b78-4f78-b4f5-ddf316d3ecf1 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:34:24.187006 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8e121eb0-1b78-4f78-b4f5-ddf316d3ecf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:34:24.200815 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e121eb0-1b78-4f78-b4f5-ddf316d3ecf1 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:34:24.256662 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e280659-161c-4974-921b-8f1319658ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:34:24.257565 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e280659-161c-4974-921b-8f1319658ceb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:34:24.257852 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e280659-161c-4974-921b-8f1319658ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:34:24.258137 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e280659-161c-4974-921b-8f1319658ceb admin admin] PATH_INFO: `/v3/users/45747dabd8254bbea4dddd514647c4e4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:34:24.259326 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e280659-161c-4974-921b-8f1319658ceb admin admin] RBAC: Authorizing `identity:delete_user(user_id=45747dabd8254bbea4dddd514647c4e4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:34:24.261268 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e280659-161c-4974-921b-8f1319658ceb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:34:24.309081 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3e280659-161c-4974-921b-8f1319658ceb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45747dabd8254bbea4dddd514647c4e4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:34:24.309751 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3e280659-161c-4974-921b-8f1319658ceb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45747dabd8254bbea4dddd514647c4e4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:34:24.317146 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3e280659-161c-4974-921b-8f1319658ceb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45747dabd8254bbea4dddd514647c4e4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:34:24.329511 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3e280659-161c-4974-921b-8f1319658ceb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45747dabd8254bbea4dddd514647c4e4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:34:24.330171 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3e280659-161c-4974-921b-8f1319658ceb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45747dabd8254bbea4dddd514647c4e4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:34:24.332134 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:34:24.344134 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1447/2904] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 4 09:34:24 2022] DELETE /identity/v3/users/45747dabd8254bbea4dddd514647c4e4 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:34:24.524191 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735b397d-7536-4019-9f82-84a297c99645 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:34:24.525751 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-735b397d-7536-4019-9f82-84a297c99645 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:34:24.538852 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735b397d-7536-4019-9f82-84a297c99645 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:34:24.560462 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:34:24.599960 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a089cc-3cde-4c5c-a3de-d4a7702739fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa43c0e65110476d9e52a93b45c1a988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a34f9b2c1cbf4bceb4067648553a5416', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:34:24.600857 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a089cc-3cde-4c5c-a3de-d4a7702739fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:34:24.601183 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a089cc-3cde-4c5c-a3de-d4a7702739fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:34:24.601467 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a089cc-3cde-4c5c-a3de-d4a7702739fa admin admin] PATH_INFO: `/v3/projects/cc1b69d242fd470690760c789911c41a` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:34:24.602657 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a089cc-3cde-4c5c-a3de-d4a7702739fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc1b69d242fd470690760c789911c41a)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:34:24.604603 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a089cc-3cde-4c5c-a3de-d4a7702739fa admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:34:24.664179 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a9a089cc-3cde-4c5c-a3de-d4a7702739fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc1b69d242fd470690760c789911c41a', 'request_id': 'req-a9a089cc-3cde-4c5c-a3de-d4a7702739fa'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:34:24.670674 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a9a089cc-3cde-4c5c-a3de-d4a7702739fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc1b69d242fd470690760c789911c41a', 'request_id': 'req-a9a089cc-3cde-4c5c-a3de-d4a7702739fa'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:34:24.677719 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a9a089cc-3cde-4c5c-a3de-d4a7702739fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc1b69d242fd470690760c789911c41a', 'request_id': 'req-a9a089cc-3cde-4c5c-a3de-d4a7702739fa'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:34:24.690223 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a9a089cc-3cde-4c5c-a3de-d4a7702739fa admin admin] The token cache is being invalidate because project cc1b69d242fd470690760c789911c41a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:34:24.690447 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a9a089cc-3cde-4c5c-a3de-d4a7702739fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:34:24.691080 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a9a089cc-3cde-4c5c-a3de-d4a7702739fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:34:24.693229 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1458/2905] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 4 09:34:24 2022] DELETE /identity/v3/projects/cc1b69d242fd470690760c789911c41a => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:34:34.343386 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:34:34.687131 dv-cinder-844563-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:37:20.395655 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e280659-161c-4974-921b-8f1319658ceb admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:37:20.397212 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3e280659-161c-4974-921b-8f1319658ceb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:37:20.467402 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e280659-161c-4974-921b-8f1319658ceb admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:37:20.526034 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eef781-eb9d-44f8-b747-807bd2cfe250 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '293c695f15c8437d81a619f840318f44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4076e316243946e980ce47386b18b28f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:37:20.526723 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eef781-eb9d-44f8-b747-807bd2cfe250 service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:37:20.527036 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eef781-eb9d-44f8-b747-807bd2cfe250 service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:37:20.527338 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eef781-eb9d-44f8-b747-807bd2cfe250 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:37:20.529283 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a3eef781-eb9d-44f8-b747-807bd2cfe250 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:37:20.540977 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eef781-eb9d-44f8-b747-807bd2cfe250 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:37:20.543271 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eef781-eb9d-44f8-b747-807bd2cfe250 service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:37:20.553579 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1448/2906] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 4 09:37:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:37:30.549329 dv-cinder-844563-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=58 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}